Quality Assurance in Multimedia Learning Materials/Evaluation

Indicators for Evaluation

V	EVALUATION
5.1 Evaluation strategy is clearly outlined.
1.	Poor	: Evaluation strategy is not outlined.
2.	Average	: Evaluation Strategy exists but covers only some aspects of the
		Evaluation framework.
3. Good		: Evaluation Strategy is clearly stated and takes into account many
		key aspects of the Evaluation framework
4.	Very good	: Evaluation Strategy is clearly stated and covers most key aspects of
		the Evaluation framework
5.	Excellent	: Evaluation Strategy is clearly stated and comprehensively covers all
		key aspects of the Evaluation framework.

5.2 Evaluation process is rigorous and as per standard procedures.

1.	Poor	: Evaluation process is not rigorous and does not follow standard
		procedures.
2.	Average	: Evaluation process follows standard procedures.
3.	Good	: Evaluation process is rigorous and follows standard procedures.
4.	Very good	: Evaluation process is rigorous, follows standard procedures and is
		well documented.
5.	Excellent	: Evaluation process is rigorous, follows standard procedures, is well
		documented and provides clear guidelines for future development.

5.3 Institutional mechanisms required for evaluation are specified.

1.	Poor	:	Institutional mechanisms required for evaluation are not specified.
2.	Average	:	Institutional mechanisms required for evaluation are specified to
			some extent but some components are lacking (e.g. mechanism for
			data collection / analysis for studying learner satisfaction is
			available, but no such mechanism for expert endorsement).
3.	Good	:	Institutional mechanisms required for evaluation are specified for
			most components, but not in detail.
4.	Very good	:	Institutional mechanisms required for evaluation are specified in detail for all components. (e.g. Mechanisms for Analysis of learner satisfaction, expert endorsement as well as fitness for purpose appropriately specified.)

5. Excellent	:	Institutional Mechanisms	required for evaluation are appropriately
		and comprehensively specified for all components and they
		contribute to preparing an evaluation report on product
		effectiveness.

5.4 Evaluation tools are objective, valid and reliable.

1.	Poor	:	Evaluation tools do not indicate any consideration given to
			objectivity, validity and reliability.
2.	Average	:	Evaluation tools indicate slight consideration given to objectivity,
			validity and reliability.
3.	Good	:	Evaluation tools indicate considerable attention given to objectivity,
			validity and reliability, but a systematic and integrated approach is
			lacking.
4.Very good		:	Evaluation tools are systematically developed with adequate and
			appropriate attention to objectivity, validity and reliability.
5.	Excellent	:	Evaluation tools are systematically developed, tested out and
			contribute in generating objective, valid and reliable data.

5.5 Provision for feedback and improvement is made.

1.	Poor	: No consideration is given to collecting feedback from any source.
2.	Average	: Slight		provision	is	made	for	collecting feedback	from	some
			sources.(e.g. provision for feedback from learners, but not from
			experts)
3. Good		: Provision is made for collecting feedback from a variety of sources,
			but no thought is given to its utilisation for improvement/modification
			of MLM.
4.	Very good	:	Provision for feedback collection from a variety of sources and
			systematic mechanism for improvement/modification based on it is
			outlined.
5.	Excellent	: Provision for feedback collection is made, improvement mechanism is
			outlined and requisite resource allocation is in place so that Feedback
			flows back into the system.

We invite your feedback, comments and suggestions by February 15, 2009. You may :

Input directly on the discussion page attached to this wikieducator page:
Send us an email to RSreedher@col.org or rvemraju@col.org or savisingh@and.du.ac.in
Send regular mail:

Commonwealth Educational Media Centre for Asia
8/4, Sarvapriya Vihar, New Delhi 110016

Please do include your name and affiliation if you would like to be acknowledged as a contributor in this process.

Work in progress, expect frequent changes. Help and feedback is welcome. See discussion page.