Online courses can be as effective as on campus courses, if they’re designed for the online setting. Trying to create a course that: achieves learning outcomes, is uniformly accessible to a wide range of learners, is engaging, and renders great student feedback is a difficult balancing act. Core challenges include:
There is no silver bullet, but we’ve developed a framework for developing engaging online courses that receive great student and instructor feedback.
Our north star is student feedback, and our core problem is understanding which course design elements reliably return improvements in feedback. To identify these standards, we've tried testing changes in courses. We've incorporated existing standards: Quality Matters, Universal Design. We've worked with instructional designers from a range of different colleges within the consortium. The result of this is a sprawling list of design standards that our instructional design team are responsible for ensuring our standards meet, but at their core is a simple framework that we keep coming back to time and time again. We call it: The Student Hierarchy of Needs.
The Hierarchy of Needs says that before a student is able to enjoy a course and even before they can achieve learning outcomes, there needs to be a foundational level of clarity. That might seem counterintuitive; most course development processes start with outcomes. Yes, you need to start with learning outcomes in order to define what the course is, but when it comes to improving a course, if a student can't read an assignment or students can't access it, interpret it, or rely on things like grading, you're probably not achieving the learning outcomes.
Once you have a base level of clarity and understanding, that’s when you can start working towards higher order functions like outcomes and engagement.
To assess course clarity, we analyze indicators such as student engagement patterns and support ticket volumes. High support requests or patterns of assignment abandonment suggest a need for deeper investigation into course materials and student interactions.
Our course design focuses on three main components: assignments, resources, and the learning management system (LMS).
For assignments, there's a lot that goes into making them clear to the gold standard we hold ourselves to.
Our Gold Standard: If a student looks at an assignment within about 20 seconds, can they understand what they need to do?
That means understanding what the deliverable is, why they need to do it, why it makes sense in the context of the course, and lastly how much work it's going to be. When we speak to students, we ask them to talk through how they engage with a course on a weekly basis. Almost every student we interview starts with these 3 questions when they look at assignments. While we have a broad range of assignments in terms of deliverables and what they are asking students to do, we try to ensure assignments are written in such a way that these 3 questions can be answered quickly.
For resources, it's really important that students understand how resources support the assignments for the week. You want students to be able to engage with resources and materials and understand how it helps them achieve the assignment for the week. When students can’t answer this question, we generally see students not engaging with the resources. This can lead to instructor frustrations because students are only using the resources where necessary and not deeply engaging with them. In the long term over the duration of the course, when there is a clear connection between the resources and assignments, we generally see good engagement.
The LMS is about the overall experience. We aim for consistency across our courses, reducing friction for students who navigate our systems. It's unrealistic to tailor our LMS to every college's individual system, but we strive to create a seamless, intuitive interface that minimizes confusion and enhances the learning process. This means making things simple, linear, and heavily standardized across courses, so students always where to look to find what they need.
Once clarity is established, we focus on outcomes. This begins with defining learning objectives that are measurable and meaningful. There are 3 steps here.
We rely on established educational frameworks, like Bloom's Taxonomy, to ensure our goals are actionable and aligned with real-world applications. Collaboration with our curriculum committee, which includes a diverse range of voices from academia and industry, ensures that our courses meet the needs of both students and the job market.
Outcomes then inform how we structure our courses, ensuring that assignments not only teach but also accurately assess these objectives. This involves a careful mapping of learning outcomes to assignment rubrics, creating a direct link between what students learn and how they are evaluated.
Now you've got a course that hopefully is clear and valuable, but it might not be fun yet. It might be repetitive. It might not be what it could be. It's not the gold standard, quite yet. Now let’s move on to the last component, which is engagement.
When we think about engagement, we think about developing something that students really want to do on a weekly basis. There isn’t a science to this one, at least from our perspective, but what works here is reading every line of student feedback that we get and working with instructors to make changes with the support of instructional designers.
It's not just reading feedback, but actually speaking to students, asking them about their experience, and getting that rich picture of what's working and what's not working. As long as the course is accessible and there's good outcomes, it’s important to give great educators at the colleges we work with the space to create meaningful and engaging assignments. If you try to create a formula when it comes to great assignment design, you're going to lose something.
When we think about how a course is performing, we talk about this hierarchy of needs at length. We talk about clarity. We talk about our outcomes. We talk about enjoyment. If we were to evaluate a course using a star system, we hope that it looks something like this with more stars at the bottom and an increasing number of stars as we iterate.
With clarity, you want to see support request tickets being low in the first 2 weeks. We survey beginning at week 2 to make sure they’re navigating the course well and work with instructors if students are getting confused.
On outcomes, you want to see at least 3 stars for a decent level of achievement of outcomes. You would expect students to say that they're finding the assignments to be relevant and valuable. You would expect them to say that generally they understand what's going on with the assignments and to be able to complete them really, clearly.
Then on the enjoyment level, 2 stars is probably below what you're looking for, but this would be a clear indication of where additional investment is needed. What's probably happening here is students aren't driven to really engage with the content. And so you generally find that students start engaging with assignments later in the week compared to other courses. LMS data would be able to provide insights here. Students' evaluations of assignments tend to hit a plateau when this is happening. We ask students on a scale of 1 to 7 with 1 being really bad and 7 being really great, how are the assignments?
If the assignments are valuable, but not particularly engaging, they tend to cap out somewhere at about 5.5- 5.6. To get into the 6s and up near 7, you need to have both valuable assignments and really enjoyable creative ones as well.
Ultimately the feedback from both students and instructors on courses using this curriculum design approach has been positive. Based on our survey results from the 2022-2023 school year, 87% of students like their Rize courses as much or better than their other online courses and 80% of students like their Rize courses as much or better than their on-campus courses. Here are a few of our favorite comments from students:
“One of the best classes I’ve taken in the past three years. You can experience a real work environment and assignments.”
“The course got me an internship with being a head coach for an esports team!”
“I think the most rewarding part of this course is getting the [Hubspot] certification at the end. Thank you”
“From start to finish this has easily been one of my FAVORITE classes, thank you guys for being so awesome!”
On the instructor side, when asked “How likely are you to recommend teaching courses powered by Rize to another professor?” by far the most common response was 10/10. When asked how strongly instructors agreed or disagreed with the statement “The course assignments prepared students to master the learning outcomes of the course” on a scale between 1 and 7 with 7 being strongly agree, 6 was the average answer. Here are a couple of our favorite points of feedback from instructors:
“It is probably the most practical course I’ve ever taught. For that reason, I enjoyed it immensely. My compliments to those who designed the course and I look forward to teaching it again.”
“The response to the assignment was overwhelmingly positive from students… It wasn’t “busy work” to them, but a chance to practice a skill they will use in the future and they recognized, and appreciated, this.”
By leveraging the online student hierarchy of needs as a framework for the continuous improvement of our curriculum, we’ve been able to develop courses for our 20+ programs that are clear, drive positive outcomes, and create engaging learning experiences for students.
All frameworks are limited, but the sign of a good framework is that it is one you come back to. We keep coming back to this because it allows us to approach any course design problem with a clear process, and while it doesn’t solve problems for us, it helps us prioritize our solutions around what matters most to the student.
The contents of this blog post were adapted from a recent webinar we presented as part of the Public Policy Week with the US Distance Learning Association. Access the webinar recording here.
