NASA Earth and Space Science Education Product Review
Review Criteria

Example Review Forms:  Formal K-12 Education, Educational Resource, Postsecondary Resource, Science Content, Informal Education 

Review Criteria Used in These Forms

  1. Materials are appropriate, complete, and effectively presented and of a high production/design quality.
  2. The product is easy to use and free from technical difficulties.
  3. Scientific accuracy - the content was developed with subject matter experts and is presented accurately.
  4. Instruction and Assessment - Materials emphasize effective instructional practices and pedagogical approaches and provide appropriate student assessment.
  5. National Education Standards (formal education products) - Instructional material for K-12 formal education support National Education Standards.
  6. Material is relevant to NASA-unique Science Mission Directorate (SMD) Content.
  7. Safety- The materials address NASA's first priority: Safety.  Risk(s) from proposed materials/equipment/protocol etc. , and risk mitigation procedures are identified. For websites targeted to children under 13, the site requires parental permission before collecting personal information.
  8. Overall Assessment
  9. Final Recommendation

       For more information on universal design, see the seven general principles of Universal
       Design found at:  https://projects.ncsu.edu/ncsu/design/cud/about_ud/udprinciplestext.htm

          Universal Design for Learning (UDL) has three main principles that are used in curriculum design:

For expansion on the elements of UDL see:

 

Criterion 1: OVERALL CONTENT - The materials are appropriate, complete, effectively presented, and of a high production/design quality.

Back to Top

Criterion 2: EASE OF USE - The product is easy to use and free from technical difficulties.

Back to Top

Criterion 3: SCIENTIFIC ACCRUACY- The content was developed with subject matter experts and is presented accurately.

 

3.A.    The material is free from content errors (e.g., scientific and mathematical inaccuracies, incorrect facts or statements, theory and fact are adequately distinguished).

3.B.    Text, figures and images do not create or reinforce misconceptions.

3.C.    The metric system of weights and measures is consistently used (e.g., Celsius, grams, liters, meters) or metric equivalents are provided. For example: 10 m (32.8 ft) or 28℃ (82.4°F).

3.D.    Specific and relevant references for further investigation/information are provided.

Back to Top

Criterion 4: INSTRUCTION AND ASSESMENT- The  materials emphasize effective instructional practices and pedagogical approaches and provide appropriate student assessment.

 

4.A.    Observable events, problem-centered tasks, and concrete experiences introduce key concepts of science.

4.B.    Learners are encouraged to explore ideas and events, to construct relationships and patterns, and to make and test hypotheses.

4.C.    Learning activities offer opportunities for students to revise or dispel prior conceptions, to build on existing knowledge and to create or construct new knowledge.

4.D.    Opportunities are provided for learners to apply concepts. Materials provide opportunities for interactive learner participation (e.g., hands-on experiences, small group activities, communicating and sharing information is promoted).

4.E.    Product is designed to foster outcome-based learning goals.

4.F.    Materials include assessment suggestions and scoring criteria that are appropriate for the type of product and/or audience, including assessments that require students to apply concepts, analyze or synthesize information, or evaluate situations (i.e., to use higher order thinking skills). Assessment of prior knowledge is included as appropriate.

Back to Top

Criterion 5: NATIONAL EDUCATION STANDARDS- Instructional materials for K-12 formal education support National Education Standards..

5.A.    The materials are well-aligned to education standards.

5.B.    Content and practices align to the standards and specific student outcomes are identified.

5.C.    In general, it is expected that NASA formal education materials will support national standards.  In some cases, it may be appropriate to also identify relevant state standards. These standards could include:

·         A Framework for K-12 Education

https://www.nap.edu/catalog/13165/a-framework-for-k-12-science-education-practices-crosscutting-concepts

·         AAAS Benchmarks for Science Literacy

http://www.project2061.org/publications/bsl/online/index.php

·         Common Core State Standards for Mathematics

http://www.corestandards.org/Math/Practice/

·         Common Core State Standards for English Language Arts

http://www.corestandards.org/ELA-Literacy/

·         International Society for Technology in Education (ISTE) – technology education standards

https://www.iste.org/standards

·         International Technology and Engineering Educators Association (ITEEA) Standards for Technological Literacy

https://www.iteea.org/39197.aspx

·         Next Generation Science Standards (NGSS)—see the note below regarding the EQuIP Rubric.

https://www.nextgenscience.org/news/equip-rubric-science-released

 

For the EQuIP Rubric: Only if the product is a larger unit of study designed for NGSS the developer should complete the Fillable pdf or Word document of the EQuIP Rubric at http://www.nextgenscience.org/resources/equip-rubric-lessons-units-science. There are additional helpful resources located at this site. This form should be submitted by uploading the completed form to the product submission form along with uploaded or linked product documents.

 

Back to Top

Criterion 6: The material is relevant to NASA-unique Science Mission Directorate (SMD) content.

NASA SMD products/programs should be centered on and draw upon NASA’s unique assets in Earth or Space Sciences: content/information (acquired through NASA science and technology programs and missions); facilities and tools (including observational datasets); or people (including NASA employees and NASA-sponsored scientists, technical and engineering experts) in at least one of the following areas: 

EARTH SYSTEM SCIENCE

NASA’s strategic objective in Earth science is to advance knowledge of Earth as a system to meet the challenges of environmental change, and to improve life on our planet.

NASA’s Earth science program seeks to answer the following questions:

These science questions translate into seven overarching science goals:

Back to Top

HELIOPHYSICS

NASA’s strategic objective in heliophysics is to understand the Sun and its interactions with Earth and the solar system, including space weather.

The domain of heliophysics ranges from the interior of the Sun, to the upper atmosphere and near-space environment of Earth (above 50 kilometers), and outward to a region far beyond Pluto where the Sun’s influence wanes against the forces of interstellar space. Earth and the other planets of our solar system reside in this vast extended atmosphere of the Sun, called the heliosphere, which is made of electrified and magnetized matter entwined with penetrating radiation and energetic particles.

To increase our understanding of the heliosphere, NASA seeks to answer fundamental questions about this system’s behavior:

To answer these questions, NASA’s Heliophysics Division is implementing a program to achieve three overarching science goals:

Back to Top

PLANETARY SCIENCE

NASA’s strategic objective in planetary science is to ascertain the content, origin, and evolution of the solar system and the potential for life elsewhere.

We seek to answer fundamental questions:

These important questions have been translated into the following science goals:

The Planetary Science Division includes programs with three major classes of mission destinations:

Back to Top

ASTROPHYSICS

NASA’s strategic objective in astrophysics is to discover how the universe works, explore how it began and evolved, and search for life on planets around other stars.

Three broad scientific questions and goals emanate from this objective.

•    How does the Universe work?  Probe the origin and destiny of our universe, including the nature of black holes, dark energy, dark matter and gravity.
•    How did we get here? Explore the origin and evolution of the galaxies, stars and planets that make up our universe.
•    Are we alone? Discover and study planets around other stars, and explore whether they could harbor life.

Back to Top

 

Criterion 7: SAFETY-The materials address NASA’s first priority:  Safety. Risk(s) from the proposed materials/equipment/protocol etc., and risk mitigation procedures are identified. For websites targeted to children under 13, the site requires parental permission before collecting personal information..

7.A.    Websites for children under 13 that collect personal information from children or general audience Websites that collect personal information from children (information that would allow someone to identify or contact the child), must comply with the Children’s Online Privacy Protection Act (COPPA).  For more information, see http://www.ftc.gov/ogc/coppa1.htm.

7.B.    The product is inherently safe.  Any potential risk areas are identified so they can be mitigated.

7.C.    When considering the adequacy of safety risk identification, communication and mitigation, the reviewer should consider whether the current educational product:

o    Confirms that this science educational product was reviewed to assure conformance with standards of applicable subject matter scientific excellence, integrity and objectivity, but that no review was done with respect to Consumer Product Safety Commission (CPSC) rules and regulations or any other safety science related law, rules or regulations. If a CPSC review or other safety review has been complted, note this in the review comments;

o    Conforms with the guidance set forth in the National Science Teachers Association (NSTA) Policy Position on Safety and School Science Instruction http://www.nsta.org/about/positions/safety.aspx  (it is not a requirement that a product included this statement, but it is a good practice);

o    Identifies the professional practice standard for science teachers set forth in document entitled “NTSA – Duty or Standard of Care” http://static.nsta.org/pdfs/DutyOfCare.pdf, by the NSTA Safety Advisory Board, April 2014, and any updates thereto;

o    Ensures that the protocol clearly notes that human subject testing criteria must be considered and comply with applicable law?  This applies if student or other human derived data are an integral part of the activity  (e.g. the use of humans test subjects, data derived from tasting, smelling, or ingesting any substance, or when any other physiological responses are evaluated).

o    Makes it clear that ultimately it is the responsibility of the science teachers and school administrators to use appropriate legal standards and better professional practices under duty of care to make the science laboratory safer.

o    Makes clear that it is the SCHOOL, SCHOOL SYSTEMS, LOCAL, STATE OR FEDERAL LAWS, REGULATIONS, CODES AND PROFESSIONAL STANDARDS that set the safety criteria and standards that apply to the science educational product design and use; and encourage the teacher to check with their local authorities to get approval for use of such science educational products in their classroom.

o    Makes clear that the science education product is provided as-is without warranty with respect the safety rules, regulations and laws.

7.D.    Are safety measures prescribed by the activity appropriate for the specified ages/grade levels? Can the relevant safety risk be communicated and managed with the target age group such that any mitigation steps can be enforced and not easily bypassed? If not, what ages/grade levels would be appropriate for this science education product?

7.E.    What, if any, safety equipment is identified (safety goggles, gloves, adequate ventilation, earplugs, dust masks, footwear, helmets, etc.)?   If properly used would the safety equipment reduce risk of injury or other damage and will the target age group likely be able to successfully use such safety equipment properly with the student to teacher ratio common to the target age group?

7.F.     Are there any attire limitations included in the instructions, for example if loose clothing could be a hazard or closed shoes are required. If so, are instructions clear that students whose attire does not conform to the safety criteria must be excluded from participation in the science education activity?

7.G.    If observing the daytime sky is involved, is instruction to never look directly at the sun included?  If observing Sun-related events are involved, are safe solar viewing apparatus provided along with training and instruction in their proper use?

7.H.    When foods are used (consumed or not), have appropriate steps been taken to address and mitigate risks for those with food allergies, and from contamination of food, intentionally or unintentionally?  Is the need for hand washing and container sterilization and other relevant food safety criteria made clear?

7.I.     Is instruction provided regarding injury mitigation when running, hiking, or other exercise, etc. is involved in the lesson?

Back to Top

Criterion 8: Overall Assessment

Assess the materials overall. Provide additional feedback on anything (good or bad) that was not addressed in the above criteria.

Back to Top

 

Criterion 9: Final Recommendation

Recommended: The product is an excellent candidate for broad distribution or availability, for example, at education conferences, on a Web site, etc.

Recommeneded as is, with revisions on next printing/edition: Note to reviewers: the material may not be re-printed/revised. Do not select this option if you believe the revisions are necessary before continued distribution/availability by NASA. (Provide notes on recommended revisions.)

Recommended--Distribution through Teacher Workshops: The product is recommended for distribution through developer conducted teacher training sessions, workshops, online workshops, or online training videos. Most teachers would require some introduction or training for effective use.

Recommended--Limited Audience: This product would be useful to limited audience. For example, very advanced users, lower/higher education level than originally identified, a limited topic area/course, a limited geographic area, etc. (Provide notes describing the audience.)

Needs Minor Revisions: The overall approach is very good, but minor revisions are needed before the product is distributed or recommended by NASA. "Minor" is not meant to indicate the revisions are not important, but that they can be made easily. The material is generally free of scientific errors/misconceptions or obvious pedagogical problems; if there are errors they can be easily corrected. These products are not required to go back through the panel review process once they have been revised.

Examples of minor revisions include:

Needs Medium Revisions: The overall approach is sound, but revisions are required before this product is distributed or recommended by NASA. Depending on the extent of revisions to be made, NASA education program managers may require these products to go back through the panel review process after they have been revised.

Examples of medium revisions include:

Needs Major Revisions: The product has potential, but major revisions are required to the overall approach before it is distributed or recommended by NASA. Products that are recommended for "major revisions" are required to go back through the panel review process after they have been revised.

Examples of major revisions include:

Not Recommended: NASA should take this product out of circulation/not release it as a Science Mission Directorate science education product. The quality is poor, the maerial is dated, or it is not relevant to NASA's Science Mission Directorate.

Back to Top

Bottom Image