38 posts tagged with "opensyllabus"

Most of Open Syllabus’s work is built on citation analysis — on the ability to determine what’s taught and what’s taught together.

But syllabi contain a lot of other information about teaching and learning. For the past two decades, the coin of the realm in many areas of higher ed has been the ‘learning outcome,’ conceived as a way to abstract from course contents to an enumerated set of learning goals. Sometimes these are very concrete and specific to a topic:

Sometimes they describe very broad sets of competencies:

Often, individual learning outcomes are parts of larger frameworks that cover the range of required knowledge for a program or degree. In other cases, faculty rely on learning outcome guidance (such as Bloom’s taxonomy) to develop sui generis outcomes for their classes. With a few exceptions, learning outcome frameworks in the US and Canada are defined locally, at the individual program level. This has the advantage of keeping faculty engaged with the frameworks and of keeping outcomes closely tied to student needs. It has the disadvantage, however, of making them useless for comparative or system-level understanding of the curriculum, and for solving problems that require such perspectives. One example of the latter that we’ve been working on is course transfer, in which the lack of common frameworks for understanding class outcomes makes it difficult to establish equivalence between classes.

There are, according to the Lumina Foundation, over 3000 learning outcome frameworks in use in the US. Despite some notable efforts to organize this vast array of material, incentives for programs and schools to pool or coordinate learning outcome development remain weak. Because learning outcomes are generally present in syllabi, however, we may be able to reconstruct the learning outcome landscape from the bottom up. A national map of learning outcomes — in fact, an international map that includes frameworks from Canada, the UK, and the EU — may be possible.

We can now pretty reliably extract the individual learning outcomes in syllabi. Our full dataset (currently 10.5 million syllabi) contains around 20 million unique learning outcomes. These can be compared semantically to generate a galaxy-style ‘map’ of the learning outcome landscape — here based on a sample of 3 million learning outcomes.

This is preliminary work that builds on relatively simple methods. But it has some interesting features. That wide grey band that runs through the humanities and social sciences suggests a lot of commonality to the learning outcomes in those fields, whereas the sharper boundaries in most STEM and practitioner fields imply more learning outcomes built around field-specific knowledge.

Overall, the graph shows enough definition to make us think that we can reverse-engineer a lot of the structure of the underlying frameworks. It won’t be a perfect map, but even a decent one would be leagues better than anything currently available and potentially make possible some of the system-level research and applications that were part of the original promise of learning outcomes.

We take a shot at that question over at Scatterplot.

The Co-Assignment Galaxy represents titles based on the extent to which they appear together on syllabi.  Each title is a dot whose size is determined by the title’s total assignment count in the collection.  This simple principle structures a very detailed map of fields, subfields, and their boundaries.  It also combines what we could call content-based and institutional ways of thinking about fields.  By content based, I mean that the layout is derived solely from similarities in the assigned contents of millions of classes, with no a priori knowledge about how those classes divide into sociology or history or physics (we added the labels later).  At the same time, we developed tools that do sort syllabi into the classificatory schemas used by universities, which reflect a more administrative and institutional view of fields. This institutional account shows up in the graph through the use of color.  A title receives a color based on its predominant field of assignment. Field boundaries and border zones are represented in the Galaxy by this interaction between spatial layout and color.

If you’d like a detailed technical dive into the OS Co-Assignment Galaxy, David McClure has you covered.

Another hi-res, giant field poster you can buy at the OS Print Store. Bring joy to the actual, aspiring, or just nostalgic architect/urbanist in your life for the holidays. Click to explore.

We’ve wanted to tease out movie rankings for a while. Movies are maybe the most passionately invested category of Open Syllabus citation data, surrounded by scholarly and popular debates and a teaching field that cuts across many fields. That makes them a rich target for a Lab.

GO TO THE MOVIE LAB

The OS Movie Lab (like the earlier Link Lab for journalism) is a navigable ranking of movies taught in college classes–in this case the 1201 movies assigned at least 20 times in the OS corpus since 2015. It sits outside the core OS toolset because we don’t have a reliable ‘Movie ID’ in our citation catalogs that would make them a searchable subcategory. Instead, these rankings are built from a hand-curated list derived from the larger OS dataset.

The rankings provide a snapshot of the thinking of hundreds of thousands of faculty about what movies to teach, drawn from millions of syllabi. Many of these decisions clearly belong to a Film Studies-centered discussion about how to teach the history of cinema. But the data comes from all fields and includes choices that reflect a wide array of teaching rationales.

Overall, the rankings present a very classical view of film canons and film studies — still oriented around the post-war American auteurs and the various European waves. Some of this reflects the national biases of the OS collection: around 55% of syllabi are from the US, 15% from the UK and another 10% split between Canada and Australia. The rankings also demonstrate the slowly-changing gender politics of the canon — and, at a step removed, of access to directorial roles. Women directed 3.5% of the assigned films released prior to 1990. For films released since 1990, that number is 13%.

The variety of teaching rationales visible in the rankings is also worth highlighting. Leni Riefenstahl is the highest-ranking woman director at #24, but is most frequently assigned in history classes, not film. Documentaries and historical dramas play roles in topical classes, and literary adaptations do well in literary studies. You can explore the teaching roles of these titles more fully in the Syllabus Explorer. The Co-Assignment Galaxy meanwhile, provides a very different perspective on the organization of film studies and film canons. Lastly, if you’d like to own a giant, beautiful poster of some of this data, we’ve got you covered.

Time for caveats about the data. OS’s collection is uneven across schools and over-represents Texas within the US; the 2.6 dataset (on which these rankings are built) is bigger and better than the 2.5 dataset but smaller and worse than the upcoming 2.7 version; the underlying citation catalogs (such as the Library of Congress catalog) are messy and often inconsistent; faculty citation practices for film in syllabi are poor; translated titles and director names often have variations; human labeling is error prone; machine learning is imprecise; and on and on. We spend a lot of time on these issues and have gotten better at handling many of them. But they still shape results in a variety of known and unknown ways. Lastly, determining a film’s country of origin becomes complex in the 1990s with the rise of international co-productions–especially in Europe. We have used the first listed country on IMDB in such cases.

The map, in other words, is not the territory. But it’s hopefully a useful guide to it.

GO TO THE MOVIE LAB