class: center, middle, inverse
# How to scale up your online workshop to reach 100 learners? ### Naoe Tatara, Radovan Bast, Richard Darst, Anne Fouilloux, Petri Jehkonen, Matúš Kalaš, Samantha Wittke --- ## We will talk about lessons learned from a "mega" workshop - Workshop: https://coderefinery.github.io/2020-05-25-online/ - Blog post: https://coderefinery.org/blog/2020/07/31/mega-coderefinery/ ## This presentation is a team effort
[coderefinery.org](https://coderefinery.org/) [@coderefine](https://twitter.com/coderefine) [coderefinery.zulipchat.com](https://coderefinery.zulipchat.com/) --- ## Introduction to the session - Carpentries Code of Conduct - House keeping - Questions about the contents -> HackMD - Questions about technical stuff -> Zoom Chat - Contents - What is CodeRefinery? (5 minutes) - A vision of reaching many people at once (with BR session; 15 minutes) - How we did it (20 minutes) - Role presentations (20 minutes) - Future outlook and discussion (10 minutes) --- ## CodeRefinery: project and funding - Funded 2016-2021 by [Nordic e-Infrastructure Collaboration (NeIC)](https://neic.no/) - Co-funding by Sigma2 (Norway), SNIC (Sweden), DeiC (Denmark), CSC (Finland), ETAIS (Estonia) - Important hubs: NTNU Trondheim, University of Oslo, KTH Stockholm, Aalto University - Trained over 700 learners [in over 30 workshops](https://coderefinery.org/workshops/past/)
--- ## CodeRefinery: audience and lessons Our **learners are not pure beginners**, they already develop code but want to become better at **collaboration and reproducibility** ### Typical workshop - Introduction to version control - Collaborative distributed version control - Reproducible research and FAIR data - Social coding and open software - Documentation - Jupyter - Automated testing - Modular code development ### More lessons https://coderefinery.org/lessons/ --- ## CodeRefinery: relation to Carpentries - Using Carpentries teaching style and format - We believe that our lessons fill a gap - Lessons are not in the Carpentries incubator because they stand as a mature unit and have open governance already (also not for novices?) - Co-funding Carpentries coordinator Nordics/Baltics ### Questions to the audience - Is there a need for more intermediate/advanced Carpentries-style material for learners who have completed a basic Carpentries training? - What topics/courses are needed? - What keeps our material from being used already? *Please answer via HackMD* --- ## A vision of reaching many people at once ![The first idea... accept everyone, stream for everyone else](https://cdn.jsdelivr.net/gh/coderefinery/carpentrycon-2020@5b611bd9fb3e771b0878c56feb9d357a385c56c0/img/inspiration-1.png) -- ![A bit later... crazy but genius](https://cdn.jsdelivr.net/gh/coderefinery/carpentrycon-2020@5b611bd9fb3e771b0878c56feb9d357a385c56c0/img/inspiration-2.png) --- ## A vision of reaching many people at once - Our previous workshops were in-person and not very large (~25 learners) - The promise of technology is to be able to reach huge numbers of people at once. But is that too impersonal? - Surely we must be able to get the best of best worlds... - With a consistent vision, yes, you can reach many people. But it requires that vision from the start, and everything supporting that value. ### Questions to the audience - How can large be more inclusive? How is it less? - What technical obstacles does it have? *10 minutes to discuss in breakout rooms, summarize here after. Please write answers in HackMD.* --- class: center, middle, inverse # How we did it ### Lectures & code-along exercises in the main room ### Get help and group exercises in a breakout room ### Bring your own breakout room ## An effective team work! --- ## Main room: Lectures & code-along exercises - Short lectures to explain a concept, and "asynchronous Q&A" in the main room - Demonstrate it on a very short example (and often repeat explanation) - Exercises more in the breakout rooms among groups - Check understanding: - surveys with simple (yes/no) questions - check speed with faster/slower buttons - ask a question and learners write down answers in HackMD - ask learners to contribute to discussion via HackMD - Sometimes make a code-along exercise and always repeat similar exercise in breakout rooms *Move on when about 70% have answered, so delegate potential misunderstanding to helpers during group exercises.* --- ## Breakout rooms: Get help and group exercises - Apply concepts previously taught to check understanding - Helpers often "taught" again to clarify concept - Pre-defined and fixed breakout rooms with 3 to 10 people in each: - larger groups were allowed when learners registered together - helpers played a key role to animate the group - encouraged one learner to share his/her screen during the exercise *First day was a bit quiet, especially in groups where learners did not know each others.* --- ## Bring your own breakout room - Learners could register as a team (3-10 persons) with a designated helper: - Registration form included a “team name” - Personalized emails with room numbers - Zoom name = “(5) Richard Darst” or “(5,H) Richard Darst” - The designated helper was very often someone who already attended a CodeRefinery in-person workshop, or someone in charge of technical support for the team *This is how we effectively scaled...* --- ## Helper training One week before the workshop: - Discuss [Tips for helpers](https://coderefinery.github.io/manuals/helping-and-teaching/) and [How helpers manage breakout rooms](https://coderefinery.github.io/manuals/breakout-rooms-helping/) - Course schedule walk-through with instructors and organizers - Exercise/breakout-room walk-through (**we need to do this better next time**) *This is how we get new people involved in the project: next generation of instructors?* --- ## Hierarchical helpers .left-column[ - Helpers - They don't have to know "everything" - They can call in other helpers - Expert helpers (more about this role later) - They support helpers - They cycle between breakout rooms] .right-column[
] *This is how we could take advantage of new helpers without worrying too much.* --- ## Required preparation, install times - With 100 people, there is very little tolerance for technical problems. - Huge emphasis: install and *test* software before, written in every email. - Very thorough setup instructions + university-specific instructions. - Made three videos about git configuration, plus "shell crash course" ([link](https://coderefinery.github.io/2020-05-25-online/#prerequisites)) - Two pre-workshop "verification times", request that everyone attends. - In the verification time, we would pair learners with helpers and have them test git and Python (not just install, but settings). - Verification times also serve as a "soft start" to practice tech early. - Credit to [Jason Bell](https://youtu.be/MzsJyOkxqv8) for the idea. - In the end, we did not have major problems.
--- ## Collaborative notes - Use of HackMD for collection of Q&A regarding lesson contents
Interactive questions and answer (among learners and instructors).
Index where we are right now, links to lessons.
Safer way to ask questions anonymously.
Easier to ask questions in an open landscape office.
A dedicated person answered to questions raised in HackMD (HackMD host).
Time schedule can be kept as an instructor can focus on the lecture and live coding.
Cleaned and published copy was useful to helpers and learners afterwards.
Stable most of the time with 50-100 users (depending on the edit history length).
Need to be careful to not overload the side channel with answers
- For short communications for more practical questions, we used Zoom chat function. --- ## Distribution of roles
--- ## Zoom room use for pre- and post-session - We opened the Zoom room 30 minutes before the scheduled session start. - Icebreaker starts 10 minutes early, ask people to come early - Also extra install help and discussion - After the scheduled session is over, we kept the Zoom room open for; - Debriefing among helpers and instructors, - Individual help by helper or expert helper, - Continuation of exercise by a team *This promoted community and let us more actively recruit for the next generation of workshops.* --- ## Streaming and recording (1/2) - If we have 100 people, interaction shifts to breakout rooms. Why can't more watch the lecture? - Zoom meetings are *not* the right format for untrusted people to watch. - Streaming is the right format: listeners isolated from presenters, no risk of damage. - Zoom can stream directly to [Twitch](https://twitch.tv) (or other services) without any other software needed. - It's a short step to recording - Recordings were very requested by learners, so they could re-watch what they learned. --- ## Streaming and recording (2/2) - [Twitch](https://twitch.tv) is a streaming platform originally focused on games, we knew it and it worked well. - We interacted with Twitch viewers via Twitch chat + HackMD. It worked fairly well. - We should have had less separation between zoomers and streamers, and generally engaged the streamers more
--- class: center, middle, inverse # Role presentations ### 1. Learner ### 2. Helper ### 3. Helper on a team ### 4. Expert helper ### 5. Instructor ### 6. Zoom host ### 7. HackMD host --- ## Learner .left-column-70[ - Petri Jehkonen, **learner**, Master student at Aalto University - Background - SW (C, Assembly) for embedded solutions - Expectation - Git (only) - Workshop arrangements from beginning to end were well managed - Challenges for learner - Outcome - Much more than Git ] .right-column-30[
] --- ## Helper .left-column-70[ - Matúš Kalaš, University of Bergen (Norway) - Assigned a **breakout room** - Leads exercise work in their room, or just **helps the learners** therein and answers their questions - **Forwards** interesting questions and difficult problems to HackMD or expert helpers - Depending on confidence with the exercises, needs only **minimum preparation**
(So awesome that instructors prepared everything so well!)
- If 2 screens or fast in switching, a helper may help **answering** in HackMD and Zoom chat
(There are also optional advanced exercises, and
advanced topics to discuss in HackMD)
- **Learns** a lot, feels useful, and has a lot of **fun** ;D ] .right-column-30[
] --- ## Learner / Helper on a team .left-column-70[ - Samantha Wittke, PhD student at Aalto University and Researcher at Finnish Geospatial Research Institute (FGI) - **Helper on a team** from FGI - Myself: in-person workshop **learner one year ago** - **Lower barrier** to interact/ask questions - Explaining topics **work-related** - **Easy to get lost** in course-unrelated topics ] .right-column-30[
] --- ## Expert helper .left-column-70[ - Radovan Bast, University of Tromsø (Norway) - Expert helper knows the material well - Cycles through breakout rooms to see how it is going - Ensure helpers are doing OK - Encourage discussion - Encourage learners to share screen in breakout rooms - Provide advice when needed - Helper can call other helpers via HackMD/chat ] .right-column-30[
] --- ## Instructor .left-column-70[ - Anne Fouilloux, Research Software Engineer at the department of Geosciences, Oslo University (Norway) - Taught from home with one screen (laptop) - Less pressure and easier to focus on the teaching thanks to CR team work (host, helpers, hackMD, etc.) - Need to have short assessments (surveys, questionnaires, etc.) frequently - Code-along sessions were sometimes too fast for many learners and often became demos - Useful to be instructor and expert helper during the workshop ] .right-column-30[
] --- ## Zoom host .left-column-70[ - Naoe Tatara, University of Oslo Science Library, Digital Scholarship Center - Dedicated for Zoom room management, including recording and streaming - Zoom-participant control - Communication with participants - Breakout room management - Important to pay attention to multiple communication channels - Details on [our blog post](https://coderefinery.org/blog/2020/07/31/mega-coderefinery/) ] .right-column-30[
] --- ## HackMD host - Richard Darst, Aalto Scientific Computing, Aalto University (Finland) - This role focuses on watching the HackMD and answering questions there. - This requires a different type of attention than paying attention to lessons, so we have a dedicated role for this. - Everyone can help with answering questions, but at least one person always dedicated to it. - Typical tasks - Answer questions - Bring up important questions to the instructor - Organize/comment on complicated answers - Add links and "meta-information" about where we are - Ensures consistent formatting, prepares it for publication --- class: center, middle, inverse ## Future outlook and Q&A session --- ## Future outlook and Q&A session - Continue with online workshops - not replacement to in-person workshops but complementary. - Advantages: more reach, cheaper to participate, possible to involve remote helpers. - Do larger online-workshops and streaming work for all audience types? (diversity, tech background, network bandwidth) - Encourage and accept observers/[lurkers](https://en.wikipedia.org/wiki/Lurker) more (better experience when streaming). - How to encourage participation and protect privacy while streaming/recording? - How can we make a federated model via the stream? We don't know yet. ### Join us: [coderefinery.org](https://coderefinery.org/), [@coderefine](https://twitter.com/coderefine), [coderefinery.zulipchat.com](https://coderefinery.zulipchat.com/)