Festival Report/Ch3BadgeLab

Badge Lab Chapter 3

“We’ve been given an accreditation system that’s broken and we want to fix in the next two days,” Phillip Schmidt told the crowd by way of introduction to Badge Lab.

What are badges and what do they have to do with education?

Joshua Gay, “programmer, activist, and community organizer” whose list of projects has included, ‘CK-12 Foundation, Textbook Revolution,The League of Technical Voters, Free Software Foundation, and One Laptop per Child,” devoted a series of blog posts to the topic before Drumbeat.:

In August Mark Surman wrote an enticing post entitled Experiment: Badges, identity and you. http://commonspace.wordpress.com/2010/08/12/badges-identity-and-you/

              [Figuring out who to pay attention to and who to work with is a big challenge in a community like Mozilla. Using the Whistler Science Fair as example, Les Orchard points out the underlying issue — we don’t have a quick way to parse through all the awesome to find out who’s good at what / who’s contributed what / who is doing things relevant to me. This is a common problem in online life overall. We don’t have an easy, portable and reliable way to represent our skills, achievements and social capital.

Over the last two months, I’ve been talking to people about this same challenge in another context — learning and education. Historically, we’ve used degrees and certificates to show what we know. This breaks down online — partly because we have no good way to show these credentials and partly because so much of our learning is now informal that degrees aren’t really relevant. People like P2PU, Remix Learning and others have come same conclusion Les has — we could use online badges to represent these things. Sites like Stack Overflow already use badges like this. We’re going to do the same for the Mozilla / P2PU School of Webcraft. Which brings me to the experiment I want to do: a digital ‘backpack’ that lets you store and display badges you pick up from many different sites across the web.

Badges can provide a good way for potential friends, collaborators, co-workers and employers to size you up. However, that’s only true if they can associate all your badges with you. You don’t want to send them traipsing around the web to look at sites like P2PU, iRemix and Badger to see your badges. Instead, you want to all the badges from these different places reliably associated with your online identity. With this in mind, I’ve been talking to Mike Hanson and others about an experiment that displays badges from multiple places as a part of the identity you build up through Firefox. Someone wanting to check you out would see something like this:

At an implementation level, this would work by storing your badges (or references to your badges) in both a personal data vault like Weave and some sort of claims system. It could work something like this:

The main ‘layers’ of this system are the 1. the badge issuer, 2. you and your online identity and 3. badge display and badge viewers. Specific to my proposed experiment are: • P2PU, Badger and iRemix -> these are places where you *get* a badge for some skill or activity. They act as ‘badge servers’ and would expose all the badges they have awarded to you in a structured and standardized way. • A personal data vault (e.g. Weave) and a claims system (Mike Hanson is working on a general system like this) -> from a user perspective, these items combine into a ‘personal badge manager’ that you access via identity tools in your browser or on a web site. • Your identity profile (webfinger?) plus social media sites like LinkedIn -> these display all of your badges and associate them with the rest of your identity.

At a practical level, we need a system like for P2PU School of Webcraft and the kind of badge platform Les is proposing for Mozilla. Connecting badges to a version of your online identity the you control also presents a huge opportunity in informal digital learning – everyone working in that space needs something like this as well.]

http://joshuagay.org/blog/?p=26 Fast forward a month and in comes me, Joshua Gay. Hello!

I’ve come on board to work on two projects, the first to help develop a badge backpack, and the second is to work with Philipp Schmidt and the Peer-2-Peer University (P2PU) to experiment with peer-based educational assessment and recognition. The relationship between the two projects is simple: some peer-based assessments on P2PU will result in students being given digital badges; these badges will then get stored in a digital backpack.

Seems simple enough, right? Well, that’s what I thought as well, until I started really looking into each of these ideas. What I discovered is that there are a great number of tantalizing questions, exciting implications, and possible implementations of badges, a badge backpack, and of peer-based assessment models. So, as a sort of status report to the community, I’m putting together a series of blog posts that I will be publishing over the coming weeks.

We will look at the use of badges historically from the military to the global Scout movements up through modern digital badges on web sites such as StackOverflow.com and MMORPGs such as World of Warcraft. Then we will spend some time trying to unravel the anatomy of a badge and what the various properties and data types might be for creating a digital badge system that can satisify the needs of many different types of users. Most importantly we will look at important use cases for the badges we hope to implement through P2PU as well as the needs and hopes of great projects such as Remix Learning. Along the way we will touch upon important topics such as identity; unravel some of what we have learned and thought about regarding peer-based assessment models; and hopefully begin presenting some storyboards and designs for our implementation of a badges infrastructure and badge backpack. We have an ambitious timeline over the coming months. By this winter we hope to have an alpha release of our badge infrastructure which will include an identity service, a badge backpack, and at least two different platforms existing as badge providers. Between now and then, though, we have a lot of exciting work ahead of us with the Drumbeat Learning, Freedom and the Web Festival just one month away where we hope to have much progress to share with our community and to use it as an opportunity to demo and test many of the concepts and ideas we have been accumulating. We have an exciting road ahead of us and I am very happy to be along for the ride!

Badges, badges, everywhere! (Badges and Assessment: part 2)

               Posted on October 15, 2010 by joshuagay            
                               

Ribbons and badges of a US Navy sailor. This is part 2 of Badges and Assessment , a series of blog posts to inform you of the work I am doing with the Mozilla Foundation and P2PU. In my quest to begin designing a badge framework, I have come across a number of fabulous examples of badge systems that span across both the physical and digital world. So, I’ve decided to share with you the top ten badge systems I have come across so far.

10. Slashdot.org This social news site awards badges for various kinds of achievements, such as The Tagger, Posted a Comment, The Contradictor, Days Read in a Row, Comedian, and Member of the {1,2,3,4,5} Digit UID Club. With a large community and a lots of data and functionality on the site, this minimal badge system has a great potential for growth.

One area I would love to see badges more tightly integrated is with the fabulous commenting and moderation system that combines properties such a User’s Karma, comment ratings, and numerous comment descriptors (e.g., offtopic, flamebait, troll, redundant, insightful, interesting, informative, funny, overrated, or underrated).

9. World of Warcraft There are hundreds of achievements you can earn that cover every facet of game-play. General gameplay achievements cover everything from the number of levels you have completed to getting a haircut. There are achievements for alliances, and player-vs-player achievements. But, perhaps what makes achievement badges so interesting in WoW is not the pervasiveness of them, but, how well they have been designed as part of the games overall infrastructure.

8. Xbox Microsoft has built into its online version of Xbox and online services a general badge system based on player achievements. What is novel about Microsoft’s badge system is that it has created, in essence, a common currency across multiple games and network services which takes in the form of Microsoft Points. Badges on the Xbox system are simple, containing just a few properties, including, a point value; a color (bronze, silver, or gold) designating difficulty/rarity; a description/image. This simplicity allows players to aggregate points from different games and services they are interacting with. To help bolster this new functionality, the site Xbox360Achievements that features both a leaderboard as well as a marketplace where you can cash in your Microsoft Points for access to various other network services.

7. Professional Association of Diving Instructors (PADI) The PADI Rescue Diver Badge PADI badges are earned by completing training courses with PADI certified instructors. Courses and badges range from fun and playful such as the underwater photography to serious and dangerous topics such as deep water diving. PADI has a fabulous curriculum and they have put a lot of effort into building an e-learning system. But, perhaps most importantly they have a fun and enthusiastic community of divers that embrace the PADI assessment and accreditation model.

6. Wikipedia Barnstars Original Barnstar We’ll let the Wikipedia page speak for itself: It is the custom to reward Wikipedia contributors for hard work and due diligence by awarding them a barnstar. To give the award to someone, just place the image on their talk page (or their awards page), and say why you have given it to them. If you are sure the barnstar is appropriate, don’t be shy!

5. Peace Corps Merit Badges The One Laptop per Child Merit Badge The Peace Corps Merit Badge project is run by a group of Peace Corps volunteers. The Web site contains a collection of badges, as well as the ability to suggest new badges or to design your own badge. What I find unique and interesting about the Peace Corps Merit Badge program is that each badge is associated with a life changing moment or a monumental triumph. I hope that the Peace Corps badges will be a way for people to conveniently be able to share their stories with others in life, whether it is a prospective employer or with next generation of Peace Corps volunteers. If you want to be inspired by one such story, I recommend reading the blog post, Peace Corps OLPC Merit Badge: Hardest Badge You’ll Ever Earn. 4. Martial Arts Rhee Tae Kwon-Do 1st, 2nd, and 3rd Dan black belts. The ranks and belts awarded in martial arts vary across arts, styles, and even within individual organizations. However, despite this, the belt system is still a fabulous example of a modern badge system that has wide spread adoption and success. The image to the left is above is an example of black belts of Rhee Tae Kwon-Do. Wikipedia informs us that this particular school has a rank system that works as follows. There are ten coloured belt grades, or kup ranks (급), and nine black belt degrees, or dan ranks (단). [... snip ...] Non-black belts, from white through to brown, denote the kup ranks. ‘Tips’ denoting odd-numbered kup ranks are marked by a stripe of the higher colour near the right end (from the wearer’s point of view) of a belt of the lower colour. Black belts denote the dan ranks. A specific dan rank is represented by the number of white bars embroidered on the black belt. One question I have about martial arts belt systems is whether belts and stripes are always earned as a recognition to a summative assessment process (i.e. tests) or if they are ever given as the result of a formative assessment process (i.e., during the normal course of instruction/exercises in class).

3. StackOverflow.com

StackOverflow.com is a community driven Q&A site. You earn badges along the way for everything from filling out your profile (“the autobiographer” badge) to providing great answers to peoples questions. Badges are organized into three categories, Bronze, Silver, and Gold, which signify their rarity and difficulty to earn on the system. Another component of the StackOverflow system is concept of “reputation”, which is a point system you earn for various kinds of actions on the site, e.g., posting, commenting, etc. Some uses of the site require a certain reputation level to be attained, for example, you can not rank up or down a given post unless you have attained a reputation of 125. The badge system coupled with the point-based reputation system creates an interesting, almost game-like nature to the site. Another site that runs on top of the same software platform is MathOverflow, which has managed to create a community that ranges from amateur mathematician to professionals (including Fields medal winner, Terence Tao). But, if you are looking to launch your own StackOverflow-like site, I recommend trying a different platform – OSQA is a free software based platform that is built on top of Django and has replicated most (if not all) of the functionality of the StackOverflow sites.

2. The Scout Movement The Scout Movement encompasses the Cub Scouts, Girl Scouts, Boy Scouts, and many other youth-based scouting organizations worldwide. A common component throughout almost all of the scout movement is the ability to earn merit badges. It is typically the case that the national branch of the organization dictates what badges Scouts may attempt to go for as well as be the sole authority when it comes to granting badges (e.g., Scouts submit applications that are submitted in conjunction with a troop leader; if the student passes, the national organization mails them an embroidered cloth badge). While researching the Scouting movement, one fabulous resource I stumbled upon was an independently run MeritBadge.org Wiki.

1. United States Military U.S. Marine Corps Gen. Peter Pace A lot of thought has been put into the awards and decorations that are awarded by the US Military. When you earn a Military medal there is a good chance you will also be given a small collection of other accouterments to put on your uniform or clothing for a variety of different situations and outfits. For example, along with a medal you could hang around your neck, you might also be given: • A miniature medal to wear on your left chest • A ribbon bar to wear on your military service uniform • A miniature medal pendant to wear on your mess dress • And a lapel badge to wear on your civilian clothes.

But, it’s important to understand that not all of the decorations on a military uniform represent a medal. For instance, many of the ribbons awarded have not medal associated with them. Further, there are also a number of other insignia and patches that serve a different purpose than ribbons and medals. Further, when looking the medals, ribbons, or other decorations you should also check to see if there are any devices attached, of which there are a wide variety. Yet, despite the depth and breadth of the system, every detail of it has been described in exacting detail, down to the exact dimensions; where and how the decorations and various medals should be worn; and what combination of medals and ribbons is appropriate to wear.

While researching the world of military badges one of my favorite resources I came across was a site by M.R. Gustafson called the Ribbon Checker, which allows a user to select boxes of various medals and ribbons earned, and then it will generate a web page containing a properly ordered and arranged rectangle of military ribbons. He has three different forms: US Navy, US Marine Corps, and the US Coast Guard. I look forward to having a similar Ribbon Checker for my educational and DIY badges I earn in coming years! If you would like more information on how you can get involved in helping us research existing badge systems, or to get involved in the design and development of our current badge infrastructure, please visit https://wiki.mozilla.org/Drumbeat/Badges.

I sat in on the first Badge session of the fest, which brought together a group of teachers and learners from mexico, argentina, us, canada, portugal, spain, india . There was a range of interests in Badges:

Dale Dougherty, from Make magazine and the Maker Faire, and Mitchell Stevens of Mozilla: “I’m interested in self taught informal learning. It’s less for me about badges and more about trying to make visible what’s happening in informal education.

I think of it as a set of paths thru various stops: How did you get to be a Welder, how did you learn to do this? What’s unique in the DIY space is the eccentric paths that people take.”

Mitchell Stevens: “I’m interested in p2pu, and how we can authenticate technology learning so that it’s a viable alternative to CISCO and MICROSOFT certification so Mozilla participants can use it to find a career,”

Ingrid Erickson (Research Fellow and Program Officer in the Digital Media and Learning Program at the Social Science Research Council) : “interested in MOBILE , informal , hard to contain modes of learning & new forms of assessment.”

Anna (College Unbound/Big Picture Learning) : “one of our biggest goals is to offer our students recognition for what they already know how to do new things they’re learning.”

Heather (Crisis Commons” “looking for ways for people to crowdsource info and software and learn that and make it packable & badgeable make it part of their daily living.”

Alex Halavais from Trinity College: “diplomas and transcripts have a chokehold on the education process. I’m interested in making Institutions and noninstitutions interoperable.

Ravi (PhD from Indiana U): “how do we validate informal learning that’s where the best learning is happening? What happens when we attach validation because the minute we attach it, someone games the system?”

And Phillip: “Figuring out badges making learning visible is the key compontent that will let us change how education works today.”

The discussion over the next two days ranged widely but settled on a few key points. One group came up with the five data points needed to define a badge:

1 – identity of receiver, 2 id of issuer 3 rationale 4 audience 5 EVIDENCE 

---

This session had 2 tracks: Badges for digital storytelling Badges for collectives

Erin's Notes on Badges for Collectives:

We’ve mostly been talking about how to badge people can we badge collectives and organizations? classify organizations so that I see these icons and I know something about them?

each person would choose the icons that they want has machine readable code on the background

how to validate? person clicks on the badge to see the information, community can vote to give +/-

what do these badges do - creates visibility for the institutions

for profit/nonprofit reproducible/exclusive - open/closed? reinforces the commons: ecology/body/town/digital representative/grassroots - hierarchical/flat

have only positive? create community-what do you want to promote? don't push people to make decisions that they don't want to make

let community create new badges? instead of trying to define all the commons badges, could have community submit or suggest commons badges. for example, I work for a nonprofit that works on open education…would love to have a badge for that

or we do research and release our data openly…so an open data badge you shouldn't have to define them all - but could let people define (or at least submit ideas on) the ways that they contribute to the commons….this does not exist at all currently nonprofits are only findable by: ratings/numbers on how much they spend and how they spend it, keyword or geography…if could discover nonprofits by contribution to commons, etc., that would be really powerful

---

Notes: "What's Wrong with Badges"

Not intended to be a complaint session, but for identifying issues and possible solutions or ways to avoid the issues as we move forward...

What's Wrong with Badges Recreating factory system of education - badging open ed - not evergreen college - no grades, just recommendation yes/no == 1/0s assessment and representation of the assessment could be badges for no criteria at all - but the assessment portfolio assessment badges linked to metadata, more information badges as conversation openers - then keeps them low-stakes less of a reason to game it ? badges represent the endorsement

badges as Yes/No

incenting the wrong behaviors, intrinsic/extrinsic motivation

open nature of badges - what happens if commercial gets in the game?

Can we make sure our badge system eats theirs? why couldn't we let other peoples badges work? what if people start making money off of this - derail it by having open operability we are free and open, we are the standard what if Bb [Blackboard, the commercial learning management system] gets into the game?

[or what if Kaplan gets into the game? As I wrote on my blog: “Kaplan (a subsidiary of the Washington Post Company whose solid profits subsidize, yes, the Washington Post: traded on the NYSE as WPO) is putting some money into an internal startup, a new assessment and accreditation business, focused on offering college credit for prior learning—including self-learning taking advantage of free Open Courseware such as that provided by MIT. This could be the long-awaited missing link for open courseware–everyone says that an open, democratic accreditation system is the Holy Grail that’s sorely missing in order for free and open courseware and peer-based learning networks to translate into affordable, accessible, higher education. Or it could be the Evil Empire taking over and strangling the edupunk movement.”

equity revolution in education infuse our thinking with collective intelligence replicating our own bias in the system - how do we include marginalized communities

badges heavily focused on getting jobs what if they are about kinship, recognition building an open framework that has minimized the forced structure not just decided on by an authority like Twitter as it evolved - the community defined how the system evolved, started very simple and community decided where they wanted it to go

categories of badges / functions quantifiable/automated - awarded by machine that you passed a certain threshold community-endorsed (qualitative?) badge infrastructure should be agnostic about how that decision is made - like internet, no idea what people would do with it, but just enough structure to support innovative use/needs badge needs an endorser - might be an algorithm, community, self too many dimensions to categorize? just need to realize that there are different types of badges - we are trying to make the equivalent of verisign - identity, verification have to have the identity behind it, 'HUB' in demo follow link to see where the badge came from badges have badges? - aggregator or mozilla-approved badges some might do very heavyweight or very light, there is weight based on the giver (page rank)

need the Technorati of badges creating an open standard for those that want to play … for it to succeed, we have to seed well executed uses of that platform, and starting points for people that want to use it the point is, what does "well" or "good" mean? that they have accomplished something, there is some reason to have a badge?

are same hierarchies going to be replicated? sense of playfulness, instead of just reinforcing the existing divides good for the open community what we are talking about is fundamentally more democratic, b/c the credibility of the badge will depend on the authority of that issuer open structure which is easy to use provides some way of saying that you can trust this badge - it resists reinventing the current system will it shift the hierarchical nature of education? if there are individuals with very high endorsement here its very relevant and important - depending on who you are interacting with, may have a lot of validity and importance institutions don't go away but there is another path… if you can prove that Harvard itself is what is making smart students - if there is that granularity, you could have a Harvard-endorsed badge without a Harvard degree.

education isn't just schools so much informal education - badges are an opportunity to capture that informal learning

badges should be binary - can see the evidence but either you have it or not issue date at all of them - learning path (when they did it and when it changed) make history transparent

summary: there are a lot of issues and concerns about what badges can be - we are agnostic about that we provide you with a way that makes it more explicit we keep it as lightweight as possible we don't have to solve all the issues b/c you use it as it needs to be lets not over-engineer it from the beginning

next steps working group that reaches out to these companies and create standards (stackflow, Facebook, 4square) framework - openness, use case and success stories so that people can look at as a starting place so people can jump in role of employer?

---

School of Webcraft Badges (Joint session with Webcraft Toolshed

Gunner: Welcome! We're going to do some tinkering around badge design. 3 small breakouts where we'll invite people in those groups to spec out how they would function.

Pippa: symantic html/css - how do we assess and give good feedback.

Christian: Accessability - how do you keep content independent of device etc.

3 questions we're thinking about: who issues and how it is earned. Who values, and how is it ..., 3rd what info is actually available with the badge.

---

Badges for Accessibility

=======

Chris: people want the badge, but what about the official badges from the w3c, which people put on their site, but didn't really do anything very useful. So, instead of applying for a badge, hunt around for really good examples, because there aren't many.

Instead, sites could be *nominated* for accessibility, ...

Also, how to visualise, different countries represent with a wheelchair, but accessibility is not about disability.

Kate, a lot of commercial websites have 'is this website helpful' which could also be used.

Christian, so there has to be a peer review of the reviewers as well. Large font. We could tie it into a firefox extension, or use a delic.io.us tag for nomination, to enable a bottom up process. Could also have a negative badge

Kate: most people don't know, because they're just beginners etc., a bit like fashion.

Daniel: who would be rewarding it. Kate: just one team, I don't see it becoming viral (reviewing them, that is). I also think they should expire given the rate of change etc. (Daniel, or rust).

Summaries: How do we do this for p2pu? Explainer, Enabler, Developer (with CC-like easy explanations).

Semantic HTML / CSS group

The Badge as a menas to demonstrate skills to interested parties

q: who provides the badge? a: maybe mozilla / open web education alliance

q: who would you like to show the badge to? a: employers, clients, students, communities, target audiences (users on the website), readers, other webcraft community members

q: what is being assessed? a: Best practices, an inclusive way of coding the website, beyond its visual appearance: Semantic HTML/CSS

steps to *earn* the badge

1. It validates!

 demonstrate a certain level of complexity, existence of metadata, alt tags and
 w3c

2. code review

 judge actual *content* as well
 audience: multiple peers
 media: questions/answers, video, documentation

 * communicate clearly what you've done
 * opportunity for others to learn from this particular project 
 * commented code
 * Annotated CPAN http://www.annocpan.org/ example of justifying design decisions, peers can annotate as well

3. Authority? Deadline, criteria. If you don't meet the criteria, you "flunk out".

• Link to Drupal process

Keep metadata? e.g. reference to actual technical artifact, source code, revision. And to the version of the standard you are validating against, and the date? If you've earned this badge, exactly *what* is being evaluated at this point in time? e.g. CSS2/CSS3

4. Renewal/validity

 If I earn the badge on 1999, it might not be equally "valid" on 2003
 Course revision
 The peer assessing community will also evolve its evaluation standards
 

• Maintain a history file as a measure of your achievements

 Also acts as a portfolio

• In the process of peer review, is it important to keep track of who have you collaborated with?

 You should definitely keep track of people who've donated time to 
 Replicate Amazon metareview system? "Was this review helpful to you?". It may be possible to game the system in this way.

• Bugzilla parallel - improve learning - file a bug? "These criteria may be out of date".

BADGE FOR SOFT SKILLS ERIN's NOTES: Session 2: Webcraft + Badges who issues it/how is it earned what data does it carry/what does it look like who values it/how is that value conveyed or passed along

SOFT SKILLS question/answering teamwork active community member communication problem solving/critical thinking talk to normal people working in decentralized way presentation/summarize documentation/communicate to nontechnical community (how is this different in an open environment, need to do the managerial role, gather specifications from a nontech audience) patience synthesis

metaphor: journalist, translator communicate technical things to nontechnical people

the form which this is expressed

title: I Talk Human; Tech Bridge (sounds two ways but that's ok?); Liaison; Bridge Badge Community Bridger

how is it evaluated? who issues it? tech and nontech people have to way in performance based describe what you have just done to local community; gather their endorsements submitting a video? have external people evaluate? non-automatic submit on the web and invite members of the community, form for submitting correspondence, community members vote based on rubric….all of that becomes the link behind the badge optional field for self-reflection

how is it earned? person submits 'proof'

value/signify? if you come across this badge in the wild, what does it signal to you? what can you do with it?

issues? unintended consequences - don't want to reinvent school private information gaming can this hold people back - young people with motivation and skills, have to get the badge to prove themselves or…if looking to hire someone and need to explain why to hire someone young or without college education…could use the badges

expiration? thumbs up/down on the badge after assigned? if have "like" button - that gives endorsements to both you and the endorsers does this open us up to gaming? or is that there anyway?

remember... doesn't have to be the only way to get badges if you don't feel like this is the way to show your skills, hack the system create your own badge and see if community adopts it

role of the creator/author of the badge validator of the badge

badges: semantic html (lots of metadata, linking back to the source, etc. check out wiki) accessibility (enable accessibility, help people understand it, developer) soft skils

anything based on technology will become obsolete so maybe badges expire, fade away, rust badges need to be validated - link to the profile and write recommendations, etc building code metaphor - can provide the codes or just say 'solve a problem'

---

Badge lab

3 sessions

1. pedagogy 2.tech 3. community \report 2 -3 main points

-anna - how can a badge be linked to a student portfolio, by a community but be able to show the work and reps the outcome -teachers , students need a reflection process, does it include a portfolio to represent it -justify the work to meet the criteria

-viability, credibility matters for community and individual effort -the group effort, the credibilty of the team effort

Tobias - kabissa connections Badge - use cases KABISSA CONNECTIONS as a case study. KC just won the Netsquared Challenge and is described here: http://netsquared.org/projects/kabissa-connections

Kabissa uses CiviCRM currently as our database platform, and we'd love to be able to issue badges along the lines of KC.

-org increase trust by revealing their trust with partners, connections and networks. -can add connections to ngo dashboards, or community boards -organizations can be grantees, opt into badge on kabissa, grantee from ford foundation (ie) -builds up trust for organizations -how do you know which orphanage in kenya to give a scholarship fund, do they have a badge, have they been vetted?

-could use for conference participation -in Africa - difficulty of getting trust -in US - paypal builds trust, 501c, audits etc -in africa - how do you know that an audit firm is legit -how to surface communities and organization -kabissa would offer the badge based on their research

Crisiscommons-heather -give badges to individuals or groups -criteria defined by x number of open source -group and collaboration - issuer describes, creates icon, they determine who is in, etc -maybe self-selecting (Kabissa) -badges are on /off crisiscommons would have a wiki link to the badge for proof of content, acheivement

1. participation 2. design has to be simple for the badges, clear 3. use it as a trustworthy measure. 4. criteria - set outcome and then back-map it, what are the stages? 5 - 10 outcomes (mini-outcomes (teacher inputs)

Kabissa - badges are a means, the process, it is a framework to make that outcome possible Set the outcomes you want to see, then backmap to the outcome. If you want the badge to represent a credential or an accomplishment, what are 5 markers of having met that accomplishment? Also, can you suggest (as a form of education, how they might accomplish each of those markers? The 5 markers need to be publicly available--if you have this badge, what does that mean you can do, what you can accomplish. Is there progress towards a badge you can show? - be able to search content db and find badge matches for ngos, individuals, by country

Report back -tobias Community As communities are looking for outcomes, change the world, show learned or accomplishments start with outcomes and back-map it look for proficiencies or credentials to lead to badge badges not user profiles, be simple, data stored in badge is not complicated - on or off.

can the markers be known - badge dependencies, can what you accomplished be publicly available?

Pedagogy -we are not new ed, eco,game theory - have great exisiting literature to base these thoughts on - commited to writing more about it and will be attending an ed conference to present on it- Digital Media Literacy conference - the DML - Alex and another badge lab attendee are already in

tech -have web dev help -will make it tell the story better, build kick ass demo -demo is big show tonight

community -start with outcomes, backmap with credentials, capacity -microformat on off format for issuers -platforms can support issuing

ERIN: PEDAGOGY AND THEORY BEHIND THE BADGE IDEA take aways: we aren't making this up, there is a lot of literature and theory that we are working with (not about badges specifically, but feedback, signalling, identity, etc.) we are committed to continuing this conversation - build a bibliography and write badge stuff together

Session 1: pedagogy/legs

       ⁃        Signaling theory - how people use credentials to signal what they know, plan to do, done in the past (economics)
       ⁃        Contracting (econ) - align interests based on those signals
       ⁃        Stian: prelim article, models for universities, sociological perspective
       ⁃        Sheep skin theory, gateway theories - sociological theories of education, and role of open education
       ⁃        Behavioral economics - feedback, incentives, entry into a group
       ⁃        Goffman - presentation of self
       ⁃        online education research: feedback, immediacy, transparency, 
       ⁃        surprise award, different kinds of motivation
       ⁃        apprenticeship
       ⁃        crowding out theory (education, organization)
       ⁃        structuration theory - how local conversations get translated into higher structures
       ⁃        gaming literature
       ⁃        visibility - signs, outward presentation (semiotics theory)
       ⁃        Wikipedia literature (barnstars)

Recognition/Certification Discoverability Motivation

---

THURSDAY OPEN THE FLOOR surprised that we didn't get the social stuff that people brought earlier can the employer be involved, valid issues around explicit value labels

soft skills - how do we credit them, and do we want to mastery in learning vs training - badges may be good for some things and not for other things, don't want to just replicate or replace what we already got

flame war issues - how to mediate them gaming - high stakes systems can get gamed, social effects will the delicate ecosystem be upset

badges as identity building, kinship less aspirational but motivational

how do we evaluate the badges and link them together into one system don't want to end up overassessing

we have a fundamentally broken system that doesn't work for most people can we improve the system and give people the recognition they aren't getting badges aren't the answer for everything but shouldn't we try? if not, the system stays the same counterpoint: degrees are still valued but scale issue - not going to hold into the future

degrees/CV are just the ticket to get into the door - can we invent something that does at least as good of a job (evidence who I am and what people think about me is hard to measure)

badges can capture vouching from peers

badges where there is a clear P/F, but that's a small subset - missing the many many other things (softer, transferable skills) dig ourselves into exactly the same hole that we are trying to get out of

1) badges as alternative to a broken system 2) how do you recognize informal learning 3) participatory, qualitative aspects <-- maybe these aren't badges but something else? differences in degree, but not in kind what are the criteria for #3?

be mindful of the communities that we are trying to help

badges: small recognitions for something that is very practical, each institution should be able to give a badge (mozilla ambassador)

challenges to bad behavior communities we want to help soft skills v hard skills

---

Introduction by:

An alternative credential arena might incentivise....

The goal standard for the value of a badge is the degree to which employers can rely on that badge as an accurate index to the .... If there is an ability to work with organizations to develop badges and certificates that have demonstrable value to the market...

Just reinforce the importance of which we watch your work in Washington.

Phillip - Reframe the question, not "When can badges replace degrees", but "How can badges replace degrees". People will break off into 4 groups to discuss the question. 20minutes, then come back with 2 or 3 points for discussion.

Session 4: What replaces the degree/report card Hal Plotkin: greetings from Obama, mentioned open education at summit in Paris... credentials: 2 ways to change HE - transform from a system that weeds people out to a system that lifts people up 1) from the inside 2) from the outside, create an alternate system - professional, technical, etc societies the gold standard for a value of a badge is the degree to which employers can rely on that badge if ability to work with expert societies to create badges with demonstrable value

not trying to replicate the grading system degrees: proxy prescribed seat time

ours: specific competencies capture the subject mastery and how - capture the path subelements - capture that, losing it now

degree by title doesn't always mean something

find out from the employer perspective, what are the criteria

existing system: society values it

badges as validation of skills and experience

not trying to hire directly to the degree, but hiring for specific pieces for the job institution decides what a credential look like, but separating out from what employer finds valuable commercial entities involved in what is education - democratize the input agnostic layer to allow external folks to express what is valuable than them

soft skills vs hard skills - people who are looking to employ graduates - if they have a voice

if we allow external communities to contribute to what is value - if we make that explicit…dangerous. danger: government has eliminated the support for humanities because they’re saying doesn't contribute to the economy

capture hard skills, soft skills, 21st century skills, etc. evidence of that through collection of badges

why are we critiquing university grading….to better capture learner (education is about employment)…better replacements for grades/degree

validation through experience

• • • making higher education a slave to capitalism

_________________

group 2: how do badges replace the report card?

what are the assumptions of what a degree is?

who are the audiences... only employers? peers? disciplines that haven't needed it: what are historical examples - medicine requires a formal system of skills and expertise - computer science/programming is more based on what i can do than a degree

how can badges stand for a reliable system of exchange - this badge = this skill/experience

what level of granularity is important? - right now: degrees are the badge equivalent - in the future: are badges equal to degrees v. individual accomplishments? - granularity can allow more specific expression of skills (not just "i am a doctor" but "i am a doctor who is has very specific expertise"

- importance of aggregated skills to be truly meaningful - what is the base unit? do granulator experiences get aggregated to mean something specific to me, or do i accumulate lots of badges in the pursuit of a specific aggregation-based badge

soft and hard skills… what are we measuring? - is there a need for "untagging" - expiration for badges that i haven't practiced lately - how about character references… but how do you measure 'leadership'

does a new badge system have legs - what proves the value? - how to convince employers this system works?

reputation is as important as skills - what does expertise really mean? who measures it? - who accredits? - when it comes to social skills… are trusted sources/recommenders more or less appropriate than "official" signoff?

let's not forget: a better system needs to prevent people from dropping out… not just a solution to reward what's already rewarded

how does the system qualify things that exist outside of formal mechanisms? - there is an intermediate kind of system in france - 'validation through experience' is perceived as a way to get "badged" that's equivalent to classical system: vetting through proven professional experience - seeing proof/portfolio/process is as important as end product skills - seeing how/what/where can help others - "knowing" v. explaining v. portfolio: how can the visibility of accumulated experience help others understand what i really know (v. a degree, which is quite opaque) - aspirational equivalents so people can see what it means

what's the role for current higher ed institutions? - it can be a new form of business for universities to to bestow badges - are they still in a position to create/set standards for expertise?

---

d------------------------

Crisis Commons example -group was very new to the badge conversation -provided an overview of the morning sessions and a crisiscommons 101. Attempted to widen the conversation to include other volunteer tech community general needs. - org needs to id the needs org can put the call out using the badge system to locate new volunteers - matching people who have the time and the capacity to volunteer could be notified via the badge system -so the value of the badge becomes a volunteer service tool in addition to individual recognition -potential for employers too

sub-note:

Very hard to hear and  concentrate in the group. I think was a product of brain exhaustion on  my part more than anything. Maybe in groups of 30 you need two note  takers and the note takers shouldn't faciliate a subsection. My own  lesson learned for how I could have helped better with documentation.

WEBCRAFT USE CASE: Session 3: Use Cases: Webcraft

web developer, badges as social innovation

distinguish between developers

badges: most badges Mozilla badges, can put together into bigger Mozilla badges our badges need badges, certified by X, issued by Y

assessment is handled by P2PU - we are developing the mechanism automatic assessment, peer assessment, course organizers

community drives the badge development, but front load it enough so that they know what badges look like webcraft community is developing the skills map - then we create badges to get this started

challenge for a badge? haven't really thought through this one

how granular should the badges be? for html development, don't want tag level badges

goals: motivation/engagement share information/skills across courses share information with the outer world (mozilla backed!!)

badge: metadata: author, even though its a mozilla badge external badging authority, spot checks that the badge

what is the ultimate goal - to get people to learn web better, or to play the badge game

peer recommendation system - 'this guy knows html 5.0' - becomes more validated as more people as soon as there is currency in the badges, people will try to game them microfinance - grameen bank, 4 other people have to recommend that person if turns out that it was fake, that person and all recommenders can't get loans yahoo answers, automated until puts up a red flag to a human build sophisticated detection, not to detect trolls, but to detect fake karma, etc need real identity something like page rank so when endorsed by mozilla

ohloh.net as a model??

Breakout Group: A badge for digital storytelling Notes, Cynthia Jimes

Initial question someone asked in our group: where in a process do you get badges? At one point in time? Is it purely an incentive? Is it an award at the end of creating something? Is it a social badge?

What are the qualities of stories that we want to capture? What are the different things we want to get out of the digital storytelling badge? What is it’s value? We may want the badge to, for example:

   * Produce an action

   * Provide enjoyment

   * Support education/knowledge transfer

   * Persuade

   * Build awareness

   * Preservation of culture 

   * Facilitate communication and engagement

   * Support co creation of meaning that translates a whole experience

   * Inspires others

Another question that arose in the group: How do you “badge” creativity? Maybe it’s about:

   * How much the story gets passed on; how much it’s shared

   * How much it’s read, accessed, downloaded

   * How much it’s remixed

   * How much it’s edited

   * How much metadata it generates (user generated metadata from others who read the story)

Other notes from our use case

   * Does the badge need an authority role, so the giver has authority. 

   * What are the levels of endorsement from those assessing the work being done

   * Reputational and economic value of the badge

   * How to think about data collection from potential badge recipients that helps us to answer the questions in our discussions and design a good badge

   * The social aspect of badges—not just for employers. Some things are harder to translate into badges, the softer stuff, like emotional intelligence

---

Badge Design Slam: Earn Your Virtual Drivers Badge.

Badge part 2: design teams Heather L

what are the requirements who gives out the badges who do you show the badge to. how do you maintain and renew it.

I joined the "how to Renew group' how to renew -how people feel about you -badges skills -using black boxes -issuance date, merge -issuing authority -how's my driving -peer review -worth measure boolean-ness of the badge is one of its greatest features- alex halavais -incentive - mayor of driving -@kabissa - free technology academy Philip schmidt -tie it to how's my driving, my insurance rate -this is my driving record in badge form - -authority with badges and acknowledge across. -i'm above average -is it peer rewarded - -anyone can be make them, low-barrier -bad with names badges -concrete, not for goal setting, open and idenitifable to -i someday want to go to mars badge- says you want to achieve. different levels of validity -can choose to reveal potentially -can you have negative badges? can you give people badges they don't want to have> do you have control -credit rating -issuing - how do they qualify, date of expiration issusing auth for id

how do you renew and maintain 1. go back to issuer, they control the badge a. renew in boolean on or off b. badges over time 2.could be part of goal setting, badges are strongest with a simple mandate 3.has to be id'ed with someone's identity, has to be verifiable

report back ideas

courtesy badge

(i was giving the report back which took my attention from proper note taking of the other reports)

Session 2: Driving badges - what types of badges would exist in this system? (12PM) 1) point of the badge system is to make people drive better so positive only badges people can decide which badges to display indirect is showing people signals to others

badges referring to skills badges referring to what type of driver you are

badge holds information about the badge, passes information to other people

basic skills - like license additional skills

get attention of the one who wants the badge, earning get attention of others, signalling

1) point of the badge system is to make people drive better positive only badges, more encouragement, avoid gaming/hiding

courtesy badge

safety things that you want people to know so that they drive safe around you warning badges: babies on board beginner driver

skills skills that you want to display automatic/gear shift driver license car gathered information in context: fast, snow, etc.

about yourself / access things about yourself that get you access to things handicapped hybrid medical

things about driver + car? environmental impact alerts within the car

driving: who issues badges? different kinds of attributes for formal (skills) v. informal (behaviors) badges - both may have subjective + objective characteristics

what's necessary in a badge system? - unbiased source - efficiency - timeliness is critical - effectiveness

right now "badge" system is: learning permit - go for it - only two kinds - what other kinds might be important in a badge system - what's the level of granularity?

what do these measure? - demonstrated competence - based on insurance demographics systems - reflexes: constellation of skills that make you drive-ready

can the car judge your level of competence (staying under the speed limit)? how might auto-programmed cars shift the focus on badges related to people driving? will cars become obsolete? the desire to multi-task may make cars go away...

how do we create a system that changes with the culture? - built-in flexibility - crowdsourcing as a means to rewards/penalize - you gain privileges as you get better/more reliable/safer? - what if social norms change between different communities

how might non-drivers participate in a badge system? - giving advice - providing highway traffic control info: where's traffic - providing info on most direct, shortest route, etc.

driving is not only an individual action: moving from individual to systemic/community-based - badges given for the benefit of the larger system - drivers as social critters - importance of badges is not for self, it's to show others that you're competent - visibility: i can see you're a good driver or not

monitoring (enforcement ) and renewal: not the same as issuing original "license" badge - what about sins of commission… are there negative badges? - can there be aspirational badges? a feedback system from the car and others - permanent v. transient badges: for knowledge v. behavior

Breakout Group: Creating a Badge System for Driving a Vehicle (Prototyping session). Notes, Cynthia Jimes

Our group worked on the question of: Whom you show the badge to and how that works

Whom do you show it to?

Show it to multiple people, and could be a lens where it’s meaningful to each group:

   * Employers

   * Rental car companies

   * Friends

   * Parents

   * Insurance Companies

How does the display of the badge work?

   * Displayed in multiple formats, access points, including display card with code to access it online, bar code. Can be an aggregate of levels of different types of driving skills.

   * Can be thought of as additional “metadata” about one’s driving skills, that is attached to the official driver’s license. This metadata can be from peers (how they view my driving), from the car (data from the automobile itself about my driving), from other entities.

   * Can be open or closed, or  selectively shown to specific people, entities

Other notes from breakout

   * Is it stored in a central repository? 

   * How is its authenticity verified? Does it need to be verified?

   * Discussion of whether the driver’s badge is an actual think that is measured at one point in time, or it is a path, adaptable, evolving, with changing links to other driving skills, information

   * Should the badge be complementary to other badge systems, so that it’s not all encompassing?

Breakout Group: What Should Go Into a Badge? Notes, Cynthia Jimes

Discussion set in the context of the group’s question of: What is a badge for? To motivate you? To show other’s something you’ve done? Could be both a recognition for achievement but also a motivating factor. Also, in the context of defining a badge as:

   * Verification/accreditation of someone’s accomplishment of a specific learning goal or other thing

   * A trajectory—things you are doing along the way, the different paths you take

   * Badges are more than a representation of some accomplishment or a trajectory of learning, they are a learning instrument themselves 

In answering what should go into a badge, the group tried to consider what “data points” would cross the three items above. These data points include:

   * An identity. Is this a self identification badge/ badge of self expression? Is it about me and my work? Is it about how my peers view my work?

   * An identity of the issuing authority. It might be an institute, a group of peers, etc.

   * A rationale or set of norms/standards. Eg, community norms, accreditation rationale

   * The audience is identified, and the badge has meaning to the audience (whether they are outside the community or inside it)

   * Having the badge link to the work that proves it, showing the learning trajectory , the evidence

Other notes from breakout

   * Go to tinyrul.com/mmporfolio for example of a learning portfolio trajectory that was developed by one of the breakout session participants [Not sure if I captured this right]

   * Badges might be organized at a metalevel at some point, so that those giving badges are certified, evaluated

http://people.kmi.open.ac.uk/sbs/2010/11/badge-lab-at-mozilla-drumbeat/ Simon Buckingham Shum Badge Lab @Mozilla Drumbeat

By sbs on November 8th, 2010

Salvador Dali Badges

What does a radically flattened, net-enabled paradigm for accrediting learning look like? As I hinted in my interview for Mozilla’s Drumbeat Festival last week in Barcelona (which was enjoyably activist/plot the revolution in vibe :-) — one of the really tough nuts to crack around open peer learning is accreditation, hence my interest in Badge Lab, where I spent quite a lot of time. (Hal Plotkin brought us greetings from President Obama, and encouragement to disrupt the accreditation status quo!)

We need to smooth the accreditation curve for informal, self-directed learning, possibly based on a lot of free internet/open educational resources and social interaction with peers. Like many educational institutions, Open U is now experimenting with the accreditation of students’ prior learning and experience in a variety of ways. But Badge Lab was pushing the model to the extreme, asking the question at the top.

We came up with myriad use cases, categories of badge, worried about whether we were in danger of reinventing the lumbering institutions we’re trying to move away from, as well as recognising, but not dwelling on, the challenge.

With due acknowledgement to the many participants who helped me clarify my own thoughts, the preliminary conclusions that I drew go something like this:

Badge Lab stickies

   * A Badge infrastructure should be  agnostic about who is awarding, and what the criteria are. Simple is best. Don’t over-engineer it. Follow the principles of net-neutrality that’s made the internet so successful. How the badge came to be awarded should sit outside the system: all the badge infrastructure needs to know is that it was awarded, with some basic, uncontroversial metadata that will allow others to build cool apps on top to browse, search, visualize, etc.
   * A Badge infrastructure is like Verisign: it gives confidence that the claimed badge really was awarded via an authenticated, trustable infrastructure.
   * A set of Badges is shorthand: something to scan quickly, a conversation opener to find out the Story Behind The Badge, just as you do when you see something on a CV, or a LinkedIn endorsement.
   * Many Badges will be Low Stakes: they’re not worth gaming the system to obtain, because their authority derives entirely from the awarder’s reputation. Since much of the time the awarder will be unknown to a third party, a key attribute of a Badge is the link the owner provides to the story/evidence.
   * There seems to be particular interest in Badges for authentic, practical learning. The point is that we already have existing systems in place for communicating and examining academic, abstract bodies of knowledge, but what’s missing is a way to document in one’s portfolio the authentic applications of one’s knowledge and skills, at an appropriate granularity. Moreover, since such displays of knowledge are by definition situated in unique contexts, the most authoritative sources of evidence will be real people who had direct contact with you.
   * We could equate Badges with clearly codified, formally assessed qualifications: a university might indeed provide a badge that confers a degree, but here we’re simply providing a digital replica of the current system. Instead, let’s think about how we appropriate symbols to project our identity. Badges could — most likely will — be used for personal expression, like your clothes, homepage, blogroll, tweets, what’s on your bookshelf or music playlist. It follows that we should think about enabling Badges to be awardable by individuals, including yourSELF should you see a way to characterise your learning from a set of authentic experiences that only you have had. I’m imagining not only community groups inventing badges to recognise talent and motivate their team, but groups of teenagers awarding each other badges, creating a local currency.

I don’t know to what extent this overlaps with what others concluded, or what’s being coded into the demo platform that was being developed while we discussed, but the sessions were enjoyable and much food for thought. Interested to track how this develops. They’re pushing forward on the coding as I write…

Categories: Events, Future Schools Tags: accreditation, drumbeat, informal learning

In the end, the group produced a working prototype as well as all this talking. This was demoed with a short video shown at the Best of the Fest:

Dan: INTRO:

CUT Shot: FAKE MARK SURMAN From behind

Narr: We have obtained hidden camera footage of Mozilla Director Mark Surman logging into the DRUMBEAT site.

(Camera comes over the shoudlder back image on his computer Ramones MARK types USERNAME: MSURMAN PASSWORD: NOONELOVESYRPERMITMORETHANIDO

Narr: But what’s this?

As soon as it logs in: CONGRATULATIONS!! YOU WON KICKASS FESTIVAL ORGANIZER AWARD!!

Narr: The Drumbeat Community has given him a badge!!

TITLE OF BADGE: THE OPEN WEB MASTER OF THE UNIVERSE. MARK accepts the badge and switches tabs, goes to the BADGE BACKPACK SITE

Narr: Mark decides to store his badge in his personal badge backpack. He wants to show it off to all his webmaster friends, so he sticks it on his personal blog.

MARK GOES TO HIS BLOG, hits REFRESH, and it’s there!

Narr: Great! Mark’s off to earn his next badge: Beer Shotgun Tutor!

Q&A

Alex Halavais :http://alex.halavais.net/about “associate professor at Quinnipiac University, where I teach in a masters program in interactive communications…a social architect, interested in ways of helping form communities of creativity, freedom, and justice..I help people to discover ways in which social media changes the nature of scholarship and learning, and allows for new forms of collaboration and self-government. ”

Q. Why did you come to Mozilla Drumbeat? A. My hopes a lot of them pin on the badge piece which is having a working system coming out of it.

I’m writing a book about motivation and measurement and I’m interested in having a kind of basic barebones system that you can throw things at and say this doesn’t work. I’d much rather there was a communit in the first part of it so it has some traction I want to award badges to people myself, and then I want to look at how they’re being awarded.

Q. Why are you interested in badges?

A. There’s 2 reasons. I like the motivation piece but leaving that aside I think it’s really important that people have the freedom to choose where they learn.

So sometimes that’s universities. I couldn’t live with myself teaching in a private university if I didn’t think universities were a great place to do that. Apple’s also great but I don’t want to be locked in to using their products.

Universities are great but if you really love the students you should be willing to set them free. Universities have to embrace that idea. We want them to go wherever learning can be effectively accomplished.

Badges are the sneaky technological back door to interoperability from one institution to another.

Phillip Schmidt “Peer 2 Peer University co-founder and director open education activist/researcher, runs projects at the University of the Western Cape and is a board member of the OpenCourseWare Consortium.”: Q. Why are you interested in badges?

A. “The idea that one monolithic institution will provide everything--teaching testing accreditation research--that idea is already an idea of the past. It seems like the institutions themselves are the last people clinging to that concept and everyone else is already innovating around them.

To define the interfaces, the fault line between formal/informal education, that's where my interest lies.

On one hand you have this existing system of accreditation it has been created for he needs of the industrial society. And now you have the possibility that you can do things differently. So at the same time if you're trying to get a job today, people will still, in many places, ask you for a degree. I think it's really interesting to see that change in open source. If you're a Linux kernel developer [an advanced member of an open source software project[ you will always get a job with whoever you want to work for.

If you have a CS degree from MIT you will, too. But if it's from a community college then maybe it's not so helpful.

These kinds of things will start happening in many more areas.

Q. People often hold up medicine as an edge case where degrees will be used for a long time. A. Certification will break out of the overall system in medicine too. There's no reason that universities should be accrediting doctors.

In the US the medical school and the hospitals they're really separate in a way i think that will happen in other areas as well.

Q. Ok, so what does all this have to do with badges?

A. something I've been thinking about quite a lot is assessment and accreditation.

Those are the crucial things where we don't have good answers yet

If you look at the Internet and the way information is managed there are a number of ways used to assess what's quality. And the CONCEPT of quality is much more individualized than it was before.

On the one hand you can aggregate opinions more than ever before, so very quickly that's a way to assess quality within a particular definition. That’s one way to look at what a student or learner is producing and what the community thinks about it.

Another way we're going to see a bit more is these trust networks.

Recently and so I've been speaking a lot with a friend of mine who has written the code fora Twitter real time search engine called Topsy about these trust networks. It's like what starts with LinkedIn: I think these are good people in that area. If you use that and place it as a filter on top of the web and look specifically at things people are doing as part of their learning, you can automate a quality assurance mechanism that's incredibly complex, because we have all this computing power. People only have to say: I trust these 3 or 4 people and you can come up with pretty accurate descriptions of learning quality, entirely taken out of the institutional space.

The second one is more, let's say I assign a certain trust value to you: I know Anya is an expert on student debt.

Even though I might not know anything about the subject in question, I can be fairly sure that there is quality there.

Q. So you replace trust in an institution’s authority, the name on the diploma, with trust in a reputation-based, peer-to-peer community? A. Well, that's future music. We do know that peer assessment works there's been a huge amount of research on students’ ability to assess each others' work. If you explain it well students can do this. There are ways to control for bias and error that can be reliable.