Formal vs Informal Education

Some people like to learn in a classroom. They are happy to sit there and have the teacher tell them what to learn, and when they get stuck, they are happy that there is someone there to help them, and they enjoy the social environment.

For many people, "education" is a synonym for "formal education", and if you tell them that your education consists of reading and learning by doing, they won't take you seriously. "Yes, but are you enrolled in any courses?"

One advantage of formal education, for those who like it, is the structured learning environment. The other advantage is that it comes with a system of qualifications. That is, you take the course, you do assignments, projects and perhaps examinations, and the results from these determine whether or not you "pass".

If on the other hand, you prefer to educate yourself informally, but you want something to show people that you have successfully educated yourself in a particular area, then you need some means of assessment which is disconnected from your choice of education process.

The Cheapness of Informal Education on the Internet

Self-education requires access to learning resources. Traditionally, the two major sources of self-learning material were books and real-life experience. And if you wanted to include interaction with other people in your educational program, you would have to find other students interested in informally studying the same topic, and if you were lucky you might find some expert willing to informally assist you in learning about the topic.

Modern technology has added a new ingredient to this mix: the Internet. The Internet can be regarded as a cheaper and improved version of the four self-education resources I just mentioned:

• Depending on the topic, enormous amounts of free written material are available on the Internet.
• If the topic is an activity which results in the creation of information, then the Internet can provide both the means to assist in the creation of information, via free and/or open software, and it provides an outlet by which you can display the results of your endeavours to anyone who might be interested.
• Via forums, blogs and general web-searching, you can find other people teaching themselves the same topic. You can write up your efforts and struggles in a blog, which may result in other people volunteering assistance and ideas.
• You can find experts on any topic, and you can even ask them for assistance. Of course an expert can be too busy to help you either online or offline, but online you have much more choice, and less effort is required to approach someone.

A big feature of the Internet is simply how cheap it is. My current Internet plan costs about the same as a book for one month's access, but in one month I am free to download the equivalent of thousands of books.

The cheapness factor is potentially even more important to people who come from poorer countries, who may simply be unable to afford entry to formal education in anything but the most basic and popular of subjects, but who may have ready access to Internet-connected computers for as little as US$0.30 per hour.

A Theory of Examinations

The traditional approach to measuring a person's level of knowledge in a topic has been the examination. These days there is often more emphasis on "internal" assessments, which may consist of assignments and projects given out by the teacher and then marked or assessed by the same teacher. However examinations remain a popular component of many assessments. Examinations have the advantages of:

• Confidence that a large number of students are all being assessed equally.
• Reduced opportunity for cheating.
• Less marking work, where an entire years assessment can be made based on the output of students over a 2 or 3 hour period.

The Internet creates opportunities for making examinations both more reliable and cheaper than they are already. To see why, we need to look at what the components of an examination process are. Ignoring the efforts of the examinees, there are three major components of effort required to administer an examination:

1. Create the examination, and keep the contents secure until the examination starts.
2. Supervise the examination – supervise the entry of students into the examination environment, identify them if necessary, ensure that they do not have prohibited materials or communication devices, make sure they do not communicate with each other, and collect the examination papers at the end of the examination period.
3. Mark the examinations.

Using computer technology, it should be possible to streamline these three phases, and also disconnect them from each other.

A Computerised Network Based Examination System

The first step in stream-lining the examination system is to computerise everything. The examinations are defined as computer files (ideally in some standardised format). The examinations are taken by the students with each student on one computer. The answers entered by the student are then retained on the computer, later processed by computer, and finally distributed by computer.

We can consider to what degree creation, supervision and marking can have their costs reduced to a minimum:

• An examination has to be created by someone who knows the subject, who puts the effort into constructing the questions and who is willing to make the effort required to keep the examination secure until the examination is take by the examinees. If an examination is one taken by a very large number of students, the cost of creating the examination per student can be arbitrarily small.
• Examination supervision is one human cost that remains proportional to the total number of examinations taken. There is a maximum number of students taking examinations that can be reasonably supervised by one supervisor. What can be done to minimise the costs of supervision is to maintain a permanent examination environment, together with personnel with expertise and experience in supervision. This can be achieved if supervision can be cleanly separated from creation and marking. For example, if 100 examinations are taken across 100 students each in 100 towns, that is only 1 student per examination per town. But if different examinations can be taken in the same examination centre, then that makes 100 students per town, with consequent economies of scale (and no requirement for the students to travel to different towns to take an examination).
• Examination marking is potentially also expensive, but this depends substantially on what kind of answers are required. Some questions have objective criteria for correctness (for example multiple choice), which means that you can write a computer program to do the marking for you. Other questions have subjective criteria for what is a good answer, for example when you have to write an essay. One way to keep costs down is to maximise the proportion of objective answers, and there are plenty of computerised examination systems which already allow only multiple choice or other types of fixed answers. There is another way to reduce the cost of answers requiring subjective marking, which is to defer all such marking (and post the answers themselves instead of the marks).

Reliability

An examination result is only useful if it can be trusted. If a potential employer judges a potential employee on the basis of an examination result, the employer needs to trust the creation process, the supervision process and the marking process.

Where these are separated from each other, the examination result must explicitly show the parties responsible for each component of the process, i.e. who created the examination, who supervised it and who marked it (or, if it was marked automatically, what algorithm was used to mark it). Anyone judging the reliability of a result must take into account all identified responsible parties.

Business Opportunity

The major business opportunity for a streamlined computer-based examination system is for the provider of the supervision component of the examination process.

The nature of the supervision service effectively determines the framework for the other processes, and also for the distribution of final results.

The services provided will fall into four main categories.

Examination Hosting Services

An examination service will provide the ability for examination creators to create examinations which can be taken by examinees on computers hosted by the examination service on specified dates. To facilitate hosting, the examination service must define standard file formats for defining examinations and for defining the marking procedures.

The hosting service will also allow the assignment of unique identifiers for all examinations. This enables the following sequence of events:

• The examination creator creates an examination, and advertises the existence of the examination directly on their web site.
• An examinee learns of the existence of the examination, and decides that they want to take it.
• The examinee makes a booking to take the examination at their local examination centre.
• On the appointed day, the examinee takes the examination.
• The examination is marked.
• The result of the examination is saved by the examination centre, which makes the result available on their website (either for public view, or for private view according to permissions granted by the examinee). The result will show who took the examination, how they were identified, which examination centre they took it at, what the contents of the examination where, what the answers given by the examinee were, and what their mark was. (In cases where subjective marking is deferred, it is up to the person looking at the result to read the answers directly and judge for themselves.)

Examination Supervision Service

The examination supervision service will maintain a fixed environment of rooms containing computers for examinees to take examinations on. The computers will be connected on a local network, and will also be indirectly connected to the Internet, to enable downloading of examinations (from creators) and uploading of final results.

Examination supervisors will identify examinees when they come in to take their examinations, and they will make sure that the examinees to do not cheat by using hidden materials or communication devices or by talking to each other.

We can say something about the details of the computer systems on which the examinations are taken:

• They will consist of computers which are booted from a known secure state at the start of each examination.
• To optimise the secure release of the examination, the examination may be pre-loaded onto the examination computers in encrypted form, with an encryption key released by the examination creator at the very last minute which is then used to decrypt the examination, putting it into a form ready to be taken by the examinees.

Marking Services

Each examination will be defined as a two-part document. The first part consists of the examination as taken by the examinee. The second part consists of instructions for marking the answers. In most cases the marking instructions will consist of a simple list of possible answers for each question, or, in the case of questions to be subjectively marked, a flag implying that that is the case. More sophisticated options may allow for simple scripts to evaluate larger sets of possible answers to some questions.

It is useful to keep the two parts of the examination document separate for security reasons, so that there is less possibility of students having any access to marking criteria while they are doing the examination.

Results Distribution Services

The final stage in the examination process is to make the results available. The most straightforward way to do this is to post results on the web. To keep everything as transparent as possible, results should be posted together with the examination papers and the marking criteria, so that everyone can see exactly what level of difficulty the examination had, how well it covered the topic, and also whether the questions were well designed.

However, it is likely that in many cases examinees would prefer to have the option of only revealing their examination results to specified parties, and this could be achieved by requiring web accounts for anyone wishing to view results, and examinees would explicitly enable the viewing of results by selected accounts on the system. There will be also be other cases (discussed below), where the creator of the examination creates it for their own internal purposes, and does not wish it to be released to the general public.

Revocations

Even where responsibilities are clearly defined, there will still be occasions where said responsibilities are not carried out, whether from carelessness, negligence or outright ill-intent. As a result, there will need to be options for people relying on examination results to question those results that turn out not to match the observed capabilities of the examinee, and, for example, where it can be determined that some particular examination centre produced many unreliable results in a certain period, it may be necessary to require examinees examined over that period to retake the examinations that they took.

Internal Examination Services

In some cases the creator of an examination will be the same organisation that is judging the results of the examination. So a company could use the services of a commercial examination service to administer and supervise a company aptitude test, which could be taken by potential employees without even requiring them to travel one particular location.

Conclusion

In many countries in the world today, education remains somewhat problematic, especially for the poorer classes. Although informal education is not for everyone, it offers flexibility and economy for at least some students. This potentially creates a demand for generic examination services, preferably with minimum possible cost, and an Internet-based computerised examination service and infrastructure is the most efficient way to satisfy this requirement.