Reading reflections

Reading Van Eck's follow-up article, I found myself nodding along because so much of it connects to my own work.

He says we no longer need to ask if games help people learn. Research shows they do. Students who learn through games perform about a third of a standard deviation better than those who don't. The real questions now are more specific: What do games teach best? How? For whom? And in what situations?

What struck me most was his point that games are especially good at building skills like problem-solving and critical thinking. He introduces "evidence-centered design," which is a way to build assessment into the game itself. This means you can see what players are learning without pulling them out of the experience to take a test.

This is exactly what I am trying to do with CivicConnect and CivicQuest. Users do civic actions on the platform, and those actions naturally show their civic skills. I do not need a separate test. The platform generates the evidence. Van Eck's framework of competency, evidence, and task models fits perfectly with what I am designing.

He also warns that early game designers made the mistake of using traditional tests to measure outcomes from games that were designed to teach complex skills. That is the same tension I face in my dissertation. Traditional assessments do not fit what CivicConnect Ghana is really building. CivicQuest must teach and assess at the same time, without breaking the flow of the experience.

Finally, Van Eck pushes back on the idea that all young people are "digital natives" who naturally understand games. Just because someone grew up with technology does not mean they know how to learn through games. That reminded me to be careful about my assumptions.

This article gave me validation and a clearer way to describe my work. It also reminded me to stay grounded and honest about what games can and cannot do.

Mishra uses the COVID-19 pandemic to ask hard questions about online learning. He draws on Neil Postman's ideas about technology. The main points are that every new technology comes with trade-offs, its benefits and harms are not shared equally, it carries hidden assumptions about how we should think and learn, it changes everything around it rather than just adding something new, and it can become so normal that we stop questioning it.

He warns against everyone rushing to use the same platforms like Zoom or Google Classroom without thinking. He argues that these corporate tools can limit how teachers actually teach.

This connects directly to my work with CivicConnect Ghana. My platform targets people who are often left out of civic education, which relates to Postman's point about unequal distribution. The platform does not just add a tool for reporting issues. It changes how users see themselves as citizens and how they relate to local government. That is the ecological change he talks about.

The hidden biases point is also important for my assessment challenge. The platform's design decides what counts as civic competence. What it rewards and tracks reflects certain assumptions. My dissertation tries to make those assumptions clear and defend them by using user behavior as evidence of civic learning.

After reading the article, I learned about a game called Crystal Island. It is used to teach science to middle school students. In the game, you play a field agent sent to a research camp on a small island. People at the camp are sick, and your job is to find out why. You walk around, talk to the camp nurse and other characters, read books and posters, and test food items with lab equipment. You write down what you find on a worksheet as you go. When you think you know the answer, you tell the nurse. If you are wrong, she tells you and you keep working.

What makes Crystal Island different from a normal game is that it is smart. The people who built it added AI on top of the game itself. One part, called a director, watches what the student is doing and decides when to step in with a hint or move the story forward. Another part tries to guess how the student is feeling, such as bored, confused, curious, or frustrated, just from what they click on and do, with no camera or sensor needed. A third part keeps a running guess of what the student seems to know as they play, and updates that guess in under a second. A fourth part tries to predict what the student is trying to do next, based on where they are standing and what they just did.

The results were strong. In one study, 150 eighth graders played the game, and the students who got full help from the AI learned more than the students who got less help. In another study with 800 younger students, science test scores went up by about a full letter grade.

Reading this made me think a lot about CivicQuest. Right now my game only changes a few numbers when a player makes a choice. Crystal Island is doing something much bigger. It is actually trying to guess what a player is feeling and what they are trying to do next, and it uses that guess to change the game while the student is still playing. I do not think I can build something this advanced on my own right now. This project took seven years and support from places like the National Science Foundation and the Gates Foundation. But it gives me something to grow toward. The part that interests me most is goal recognition, guessing what mission a player is working on from their actions. That feels like something I could try in a small way later, once the basic game is solid. For now, this article is a good reminder that stealth assessment can go much deeper than just moving five bars up and down.

McDaniel and Telep lay out ten practical habits for bringing games into an online course, drawn from instructors who had actually tried it. Reuse what already exists instead of building a game from nothing. Let students make things rather than only play things. Do not plan every minute of a session, since some of the best learning comes from leaving room for the class to wander off script. Simple games count too, a deck of cards or a board game works as well as anything digital. Keep every game tied back to a real learning goal instead of using technology for its own sake. Tell students why they are playing before they start, and talk through what happened once they are done. Mixing students from different majors tends to produce richer group work. Games built specifically to teach, not just to entertain, earn their place in a course. Tools like wikis and shared virtual spaces help groups organize themselves. And test a game on a small group of real students before using it with a whole class, then keep revising it.

I agree most with the first one, reuse what already exists, and it is honestly the one I have had the hardest time following. Building CivicQuest from scratch has been slow and stressful, and there were points where I wondered if I should have started from an existing game template instead of a blank Unity project. This article is a reminder that starting from scratch is a choice, not a requirement, and that choice costs time I could be spending on the parts of the game that actually need my attention, like the missions and the dialogue.

The tip about testing with real students before using a game with a whole class matters most for where CivicQuest is right now. The build is still small, one district and a handful of missions, and I have not put it in front of real students yet. Before I ever use it in a classroom or bring it into my dissertation study, it needs a round of testing with people who are not me, so I can see where it breaks and where it is actually fun.

Shelton and Wiley argue that when instructional designers try to copy commercial games, the copies usually fail, and it is not because the designers are lazy or careless. Commercial games work because they create real challenge, personal interest, which the authors call proclivity, and genuine uncertainty about what happens next. Educational games kill that in two specific ways. They state the learning objective too plainly, right there on screen, which breaks the sense of being inside a story. And they stop the game to ask a quiz-style question, which breaks the flow the same way a pop quiz would break a movie.

Their fix is simple to say and hard to do. Instead of bolting the learning goal and the assessment onto a game after it is built, bury them inside the story itself. Their example of fixing a motor works because the fix matters to the mission, not because a menu told the player it was time to practice engine repair, objective three.

This is the exact tension I am living with in CivicQuest. My first instinct was to build something closer to a civic quiz with graphics, and I am glad I caught myself before I went too far in that direction. The fun in the current build does not come from a menu that says civic lesson. It comes from the ordinary business of getting by: taking on small jobs, driving a taxi or a trotro route, delivering for someone, earning enough to actually feed yourself and keep going. The civic learning rides along inside that, the same way Shelton and Wiley describe. A player is not stopped and told that they just practiced honesty. They just find out that the trader remembers them, or that the wrong choice made a route harder to run. I am still working on getting that balance right, and this article gave me language for what I am actually trying to protect, which is that the fun has to survive contact with the lesson, not the other way around.

Vlachopoulos and Makri (2017) reviewed studies on games and simulations in higher education. They found that games help college students in three main ways. The first is cognitive: students gain knowledge, understand the content better, and can apply ideas. The second is behavioral: students work better in teams, communicate more clearly, and perform better on real-world tasks. The third is affective: students are more engaged and motivated, feel better about the subject, and feel less anxious. Most studies looked only at cognitive outcomes, but the behavioral and affective gains showed up just as often.

They also found that games worked best when they were mixed with other teaching, such as debriefing, reflection, and group discussion. Games did less well when they were used on their own to replace teaching.

This matters to me because it gives me evidence from higher education. Most reviews of game-based learning, like those by Clark and colleagues or by Van Eck, mix K-12 and college students together. This review looks only at college students, which is the group I work with.

The three categories also give me a simple way to plan the outcomes for our group's descriptive statistics game. It could lead to cognitive gains, like better test scores. It could lead to behavioral gains, like students using statistical reasoning in real situations. And it could lead to affective gains, like less statistics anxiety and a better attitude toward the subject.

I also like the point that games work best as a supplement, not a replacement. Our game should not try to replace the whole statistics unit. It should support the teaching that is already there.

Reading Chapter 2, what stood out first was how simply the authors state the problem. The world has changed, but schools have not. We are still teaching in a technology-rich world with books, paper, pencils, and blackboards. It is like training someone to drive a car by having them practice on a horse.

The authors call this shift the Knowledge Revolution. The Industrial Revolution multiplied human physical power with machines. Computers now multiply mental and intellectual capability. Students no longer need to memorize everything; they need to know how to find information, evaluate it, and use it. Technology has also changed what community means. People used to be connected mainly by living in the same place. Now they can form communities of interest with people they will never meet in person, and that demands new kinds of literacy beyond reading and writing: video, images, forums, social networks, and collaborative spaces.

The part that mattered most to me was what the authors say technology can do that a traditional classroom struggles to. Just-in-time learning lets someone learn a skill exactly when they need it, instead of memorizing it years in advance in case it is useful later. Customization lets learning fit the person instead of forcing everyone through the same curriculum at the same pace. Learner control shifts some of the say over what, when, and how to learn from the institution toward the student. Interactivity gives immediate feedback when a student makes a decision or a mistake, instead of waiting for a grade.

Simulations and games stood out especially. A simulation lets a student ask "what happens if I do this?" and actually see the consequence. A game can put a student in a role that calls for problem-solving, strategy, and decision-making, and can produce the kind of deep engagement the authors call flow, something a worksheet rarely does.

By the end of the chapter, the authors describe what school could look like instead: students working on real problems, like water pollution or GIS mapping, with technology and a teacher who coaches rather than lectures. That made me a little uncomfortable, in a useful way. It named something about traditional schooling, the sitting, the memorizing, the tests that measure recall rather than use, that can quietly kill curiosity. Their alternative looks more like an apprenticeship: a meaningful task, with someone more experienced guiding you through it.

Reading Ren's chapter changed how I think about measuring learning inside a game. The idea is simple to state: instead of stopping a game to give a quiz, you build the assessment into the gameplay itself. A player never sees a test screen. They are just playing. Behind the scenes, the game is watching how they solve problems, what choices they make, and how they work with others, and all of that becomes evidence of learning.

Ren frames this against a traditional test. A quiet room, a time limit, pressure to remember what you studied. That is not how anyone actually uses knowledge in real life. A game puts a player under a different kind of pressure, deciding what to do next, working with other people, solving a problem inside a situation that keeps changing. That is closer to how learning actually gets used.

The chapter is built around Evidence-Centered Design, which asks four questions in order. What skill am I trying to measure? What tasks inside the game would actually reveal that skill? What specific player actions count as evidence of it? And how do all of those small pieces of evidence combine into one meaningful score? Each question sounds obvious on its own. Answering all four for a real game, and getting the combination right, is the hard part.

Ren reviews three games built this way. One measured teamwork by tracking how players shared resources and coordinated with each other. Another measured problem-solving through a modified version of Plants vs. Zombies. A third measured creativity through physics puzzles. In all three, the hidden score matched how the same players did on a separate, traditional test of the same skill. That match is what makes the invisible measurement trustworthy rather than just a guess.

This is the same idea Van Eck pointed to in the first reading I did for this course, competency, evidence, task, and reading Ren made it concrete in a way Van Eck's article did not. With CivicQuest, I have been thinking about learning outcomes the way I was taught to: a lesson, then a check for understanding. Ren's four questions gave me a sharper way to ask what I am actually doing. If a player verifies information before reporting an issue, that is evidence of critical thinking. If they talk to more than one person in the community before acting, that is evidence of collaborative problem-solving. The game can track both without ever stopping to ask a single quiz question.

Two things in the chapter keep me honest about this. Building real stealth assessment takes a team, someone who understands games, someone who understands assessment, and someone who understands the subject, usually three different people, not one. Right now that one person is me. The other is ethics. Ren raises the question directly: if a player never knows their choices are being scored, is that fair? For CivicQuest that is not hypothetical. If this game becomes part of my dissertation study, the people playing it need to know their choices are data, even if the game itself never interrupts them to say so. Stealth inside the game and informed consent outside it have to both be true at once, and I do not think that is a problem I can design my way around. It is one I have to be upfront about.

The lesson I am taking from this chapter is a simple one. The game is not the goal. The learning is. A game is one way to get evidence of that learning without breaking the experience to go looking for it, and stealth assessment is the part of the design that decides whether that evidence can actually be trusted.

After reading Eynon's short editorial, only four pages, I was surprised by how much it is doing in that space. Big Data means data sets so large that normal software cannot handle them, the kind that can be searched, combined, and cross-referenced at a scale no single researcher could manage by hand. Everyone from governments to businesses is excited about it, and in education that excitement has produced two growing fields, Educational Data Mining and Learning Analytics, both trying to use that scale to understand how students learn. Eynon's worry is not that this is a bad idea. It is that most of the conversation around it stops at efficiency, using data to deliver education better, track performance, evaluate schools, without stopping to ask what that actually costs.

She names three problems, and the one she spends the most time on is ethics. Big Data can predict which students will drop out before they even start a course. It can recommend classes the way Amazon recommends books, based on what similar students did. It can flag a school as failing and track nearly everything a student does while enrolled. None of that is hypothetical anymore. What stopped me was her next question, not whether we can do this, but what we do once we have. If a student is predicted to fail, do we warn them, support them, or let them find out the hard way. What happens to the ordinary surprise of learning something you did not expect to, once every choice is pointed at you by an algorithm first. Eynon's point is that education has not actually had this argument out loud yet, even though the tools to do it are already in use.

The second problem is about what the data can and cannot say. Big Data is good at showing what people did. It is much weaker at saying why. Eynon's own example stuck with me: young people told researchers in a survey that they had stopped using the internet, but in follow-up interviews it turned out they still logged on now and then for a specific errand, checking Facebook, looking for a job. They just did not count that as "using the internet" anymore because it had stopped mattering to them. A Big Data trace of their activity would have shown logins and missed the real finding entirely, that the internet had quietly become irrelevant to their daily life. Data tells you what happened. It does not tell you what it meant to the person it happened to.

The third problem, inequality, is the one I think gets skipped over most often. Big Data research tends to capture people who are already online a lot, which means wealthier and more connected groups are overrepresented and everyone else is simply missing from the data set. The data itself usually belongs to a private company, Twitter, Google, a particular platform, so access to study it is not open to just anyone. And reading it well takes a specific, specialized skill set, which narrows who gets to participate in the research even further. Eynon's conclusion is blunt: a tool that could help close educational gaps can just as easily end up reproducing the same gaps it claims to study.

Reading this next to Ren's chapter on stealth assessment made the connection obvious. A game like CivicQuest produces exactly the kind of trace data Eynon is describing, every choice, every path through the dialogue, logged at a scale I could never collect through interviews alone. Her ethics question lands on my dissertation directly, since CivicConnect Ghana is built to track civic reports and engagement for people who often have the least say in how platforms like this are designed. And her inequality point is not abstract for me either. The citizens least likely to show up reliably in my data are probably the ones with the least reliable phone or data access, which in Ghana is not a small or random slice of the population.

What I am taking from this is not a reason to drop the data side of my work. It is a reason to keep pairing it with something slower. Eynon is not arguing against Big Data. She is arguing against treating its scale as proof of its accuracy. A bigger data set is not automatically a better answer, and in CivicConnect's case, the numbers on a scorecard will always need the interviews and the focus groups sitting next to them to say what those numbers actually mean to the people behind them.