Exploring Computer Science Education Trends at SIGCSE Conferences

What Are Computer Science Educators Interested In? The Case of SIGCSE Conferences Ragonis Noa, Beit Berl College & Technion Israel Institute of Technology Hazzan Orit, Technion Israel Institute of Technology

Rational One of the challenges facing CS educators is coping with the rapid changes and developments taking place in: the discipline of CS the students' interests and skills new approaches and methods of teaching, learning and evaluating These changes require CS educators, to constantly adjust their course syllabi as well as their teaching methods in order to prepare students for the unknown future studies and future work market.

Objective To explore the changing in the fields of interest of the computer science education (CSE) community. We choose SIGCSE conferences as a first case study. We reviewed four conferences between 2006-2018: 2006, 2010, 2014, 2018 Our assumption was that changes will be expressed less in subsequent conferences.

SIGCSE The SIGCSE Technical Symposium is the largest computing education conference worldwide organized by the ACM organization. It attracts over 1,500 researchers, educators, and others interested in improving computing education in K-12 and higher education. It is a forum for educators and researchers to share new results and insights around developing, implementing, or evaluating computing programs, curricula, and courses. The 2019 conference celebrated its 50thmeeting.

SIGCSE main venues Papers Panels Special Sessions Posters

SIGCSE venues Main Venues appeared at all four conferences Additional venues which arose during the years Demos (environments) Papers Panels Lightning talks (short presentation) Special Sessions Birds of a Feather (discussion of similar interests) Posters Nifty assignments (share successful assignment) Pre & Post event Workshops

Our research focused on Papers describe an educational research project, classroom experience, teaching technique, curricular initiative, or pedagogical tool. Papers Panels Panels present multiple perspectives on a specific topic. Special sessions include a seminar on a new topic, a committee report, or a forum on curriculum issues. Special Sessions

Data base Number of papers, panels, and special sessions presented at SIGCSE 2006, 2010, 2014 and 2018 Type of presentation 2006 2010 2018 2014 Papers 107 103 105 162 Panels 14 12 14 19 Special Sessions 10 12 12 16 Total = 576 121 127 131 197

Data analysis For each item in each presentation category, we examined: Session head Presentation title Categories and subject descriptors (in 2006, 2010, 2014) General terms CCS concepts Keywords (in which the papers were presented) (in 2006, 2010, 2014) (in 2018)

Data analysisconsiderations (I) Session heads: reflect the conference chairs' perspective on the conference topics and focal themes. Hence are relevant to content categorization. General terms: did not lead to any valuable conclusion. Hence, this analysis is not presented.

Data analysisconsiderations (II) Categories and subject descriptors: (link) Till the 2018 conference, most presentations were classified as K.3.2 - Computer and Information Science Education category. Hence, it was not useful for the investigation. Inconsistency: Some authors used only a high-level category, while others attributed several subcategories. The new CCS categories is a valuable tree of concepts which allows more accurate and up-to-date characterization of the presentations. (link) Nevertheless: some authors chose concept that appears as a leaf of the categorization tree, other stop at inner nodes, and some describe the complete paths. Therefore, comparative analysis may not be accurate.

Data analysisconsiderations (III) Assumptions: Keywords express themes as defined freely by the authors; selected by the authors and therefore express their presentation content. Findings: Maybe But In fact, the diversity and uniformity is wide, and therefore it is difficult to draw conclusions

Data analysis We analyze the data according to two aspects. Each item was characterized according to two classifications 1) Content analysis 2) Keywords analysis The categories in each emerged from the data analysis.

Content analysis Considering the session head in which the presentation was included, the presentation title, and if needed, its abstract, we defined: five content categories three of them have sub-categories Each presentation was categorized into one category according to its identified leading content.

Content analysis The five main categories are: 1. Teaching methods 2. Curricula 3. CSE research 4. Recruitment and retention 5. Educators (with five sub-categories) (with three sub-categories) (with two sub-categories)

Keywords analysis The original keywords were divided into three themes. When authors did not specify keywords, we retrieved keywords from the presentation title. The three main categories are: 1. Curricula 2. Topics 3. Pedagogy (e.g., CS1, K12, CT) (e.g., data structure, recursion) (e.g., visualization, gamification)

Relative frequency of the content categories of papers Content categories/ Sub-categories 2006 N=107 2010 N=103 2014 N=105 2018 N=162 Total N=477 Teaching Methods 47% 47% 53% 37% 45% Computerized tools Active learning Evaluation Projects/peers 9% 20% 19% 9% 14% 15% 12% 20% 7% 13% 8% 10% 6% 9% 8% 8% 5% 5% 7% 6% Class organization Curricula Undergraduate K-12 Interdisciplinary CSE Research 6% 0% 4% 4% 4% 22% 31% 27% 28% 27% 8% 8% 6% 21% 21% 7% 3% 14% 21% 1% 6% 9% 15% 5% 9% 15% 16% 5% 6% 15% Recruitment and Retention CS Educators K-12 Undergraduate 6% 6% 11% 9% 8% 4% 4% 0% 2% 1% 1% 0% 0% 0% 11% 6% 5% 5% 3% 2%

Relative frequency of the content categories of papers Content categories/ Sub-categories 2006 N=107 2010 N=103 2014 N=105 2018 N=162 Total N=477 Teaching Methods 47% 47% 53% 37% 45% Computerized tools Active learning Evaluation Projects/peers 9% 20% 19% 9% 14% 15% 12% 20% 7% 13% 8% 10% 6% 9% 8% 8% 5% 5% 7% 6% Class organization Curricula Undergraduate K-12 Interdisciplinary CSE Research 6% 0% 4% 4% 4% 22% 31% 27% 28% 27% 8% 8% 6% 21% 21% 7% 3% 14% 21% 1% 6% 9% 15% 5% 9% 15% 16% 5% 6% 15% Recruitment and Retention 6% 6% 11% 9% 8% CS Educators K-12 Undergraduate 4% 4% 0% 2% 1% 1% 0% 0% 0% 11% 6% 5% 5% 3% 2%

Papers CS Educators 5% Recruitment and Retention 8% Teaching Methods 45% CSE Research 15% Curricula 27% Total N=477 45% 27% 15% Content categories Teaching Methods Curricula CSE Research Recruitment and Retention CS Educators 8% 5%

Papers: Teaching Methods Class organization 9% Computerized tools 31% Projects/Peers 13% Evaluation 18% 45% Teaching Methods Computerized tools Active learning Evaluation Projects/Peers Class organization Total 14% 13% 8% 6% 4% Relatively 31% 29% 18% 13% Active learning 29% 9%

Papers Interdisc- iplinary 22% 27% Curricula Undergraduate K-12 Interdisciplinary Total 16% 5% 6% Relativly 59% 19% 22% Undergr- aduate 59% Total N=477 45% 27% 15% Content categories K-12 19% Teaching Methods Curricula CSE Research Recruitment and Retention CS Educators 8% 5% Underga- rduate 40% 5% K-12 60% CS Educators K-12 Undergraduate Total Relativly 60% 40% 3% 2%

Relative frequency of the content categories of all presentation types Content categories/ Papers Total N=479 Panels Total N=59 Special sessions N=50 Teaching Methods 45% 40% 30% Curricula 27% 35% 36% CSE Research 15% 10% 18% Recruitment and Retention 8% 10% 12% CS Educators 5% 5% 4%

Relative frequency of the content categories of all presentation types Content categories/ Papers Total N=479 Panels Total N=59 Special sessions N=50 Teaching Methods > > < = 45% 40% 30% Curricula 27% 35% 36% CSE Research 15% 10% 18% Recruitment and Retention 8% 10% 12% CS Educators 5% 5% 4%

Relative frequency of the content categories of all presentation types Content categories/ Papers Total N=479 Panels Total N=59 Special sessions N=50 Teaching Methods 45% 40% 30% Curricula 27% 35% 36% CSE Research 15% 10% 18% Recruitment and Retention 8% 10% 12% CS Educators 5% 5% 4%

Several observations Although the objectives of the three kinds of presentations differ, the different presentations address practically the same topics. And this, at more or less, with similar related frequencies. The dominance of the content category Teaching Methods is noticeable, especially compared with Curriculum. Although teaching methods receive a great deal of attention, only a small proportion of presentations address the CS educators themselves.

Keywords analysis 1. Curricula 2. Topics 3. Pedagogy (e.g. CS1, K12, CT) (e.g. data structure, recursion) (e.g. visualization, gamification) This categorization appears as similar to the type of papers specified in the SIGCSE 2019 submission site: (1) Curriculum Topics; (2) Computing Topics and Education; and (3) Experience Topics. The keywords analysis was done in relation to papers

Keywords that address curriculum 2006 N=131 16 8 13 4 4 2010 N=127 18 8 15 3 4 2014 N=131 14 5 16 3 8 2018 N=197 16 3 17 5 1 Total N=586 64 24 61 15 17 Keywords Curriculum/Curricula CS0 CS1 CS2 Software Engineering STEM/ Interdisciplinary/ Multidisciplinary Computational Thinking K-12 AP K10 Middle School Elementary/Primary/K6 Percentage in the conference 2 5 7 4 18 7 6 6 7 20 28 3 1 2 5 1 10 11 3 1 2 4 1 37% 52% 54% 37% 44%

Keywords that address curriculum Advance curricula is presented less 2006 N=131 16 8 13 4 4 2010 N=127 18 8 15 3 4 2014 N=131 14 5 16 3 8 2018 N=197 16 3 17 5 1 Total N=586 64 24 61 15 17 Keywords Curriculum/Curricula CS0 CS1 CS2 Software Engineering STEM/ Interdisciplinary/ Multidisciplinary Computational Thinking K-12 AP K10 Middle School Elementary/Primary/K6 Percentage in the conference 2 5 7 4 18 7 6 6 7 20 28 3 1 2 5 1 10 11 3 1 2 4 1 37% 52% 54% 37% 44%

Keywords that address curriculum 2006 N=131 16 8 13 4 4 2010 N=127 18 8 15 3 4 2014 N=131 14 5 16 3 8 2018 N=197 16 3 17 5 1 Total N=586 64 24 61 15 17 Keywords Curriculum/Curricula CS0 CS1 CS2 Software Engineering STEM/ Interdisciplinary/ Multidisciplinary Computational Thinking K-12 AP K10 Middle School Elementary/Primary/K6 Percentage in the conference In relation to all frameworks 2 5 7 4 18 7 6 6 7 20 28 3 1 2 5 1 10 11 3 1 2 4 1 37% 52% 54% 37% 44%

Keywords that address curriculum 2006 N=131 16 8 13 4 4 2010 N=127 18 8 15 3 4 2014 N=131 14 5 16 3 8 2018 N=197 16 3 17 5 1 Total N=586 64 24 61 15 17 Keywords Curriculum/Curricula CS0 CS1 CS2 Software Engineering STEM/ Interdisciplinary/ Multidisciplinary Computational Thinking K-12 AP K10 Middle School Elementary/Primary/K6 Percentage in the conference 2 5 7 4 18 7 6 6 7 20 28 3 1 2 5 1 10 11 3 < 1 2 4 1 37% 52% 54% 37% 44%

Keywords that address curriculum

Keywords that address CS topics 2006 N=131 5 4 1 1 4 2010 N=127 3 5 2014 N=131 3 1 2 3 1 2018 N=197 8 4 8 2 Total N=586 19 14 11 11 10 Keywords Algorithms Networks Cybersecurity Parallel Object Oriented Compilers Data Structures Web Graphics Python Distributed Functional Percentage in the conference 5 5 5 3 1 1 10 1 3 5 1 1 4 4 1 1 4 4 4 1 1 1 2 4 10 9 7 7 6 5 1 1 23% 22% 19% 18% 20%

Keywords that address CS topics 2006 N=131 5 4 1 1 4 2010 N=127 3 5 2014 N=131 3 1 2 3 1 2018 N=197 8 4 8 2 Total N=586 19 14 11 11 10 Keywords Algorithms Networks Cybersecurity Parallel Object Oriented Compilers Data Structures Web Graphics Python Distributed Functional Percentage in the conference < > 5 5 5 3 1 1 10 1 3 5 1 1 4 4 1 1 4 4 4 1 1 1 2 4 10 9 7 7 6 5 1 1 23% 22% 19% 18% 20%

Keywords that address CS topics 2006 N=131 5 4 1 1 4 2010 N=127 3 5 2014 N=131 3 1 2 3 1 2018 N=197 8 4 8 2 Total N=586 19 14 11 11 10 Keywords Algorithms Networks Cybersecurity Parallel Object Oriented Compilers Data Structures Web Graphics Python Distributed Functional Percentage in the conference 5 5 5 3 1 1 10 1 3 5 1 1 4 4 1 1 4 4 4 1 1 1 2 4 10 9 7 7 6 5 Paradigms? So restricted? 1 1 23% 22% 19% 18% 20%

Keywords that address CS topics

Keywords that address pedagogy 2006 N=131 2010 N=127 2014 N=131 2018 N=197 Total N=586 Keywords Assessment/Evaluation/Testing/ Automated Testing Teaching/Instruction Collaboration/Peer/Pair Gender/Diversity/Minorities/ Culture Active learning Research 9 13 19 36 77 6 8 17 18 9 9 29 25 61 60 15 13 13 19 60 11 11 6 8 8 7 16 9 41 35 Visualization Project Games Distance Learning /Flipped Class /MOOC Percentage in the conference 6 8 6 10 5 8 6 8 4 7 7 9 29 28 27 2 3 8 10 23 63% 80% 69% 85% 75%

Keywords that address pedagogy 2006 N=131 2010 N=127 2014 N=131 2018 N=197 Total N=586 Keywords Assessment/Evaluation/Testing/ Automated Testing Teaching/Instruction Collaboration/Peer/Pair Gender/Diversity/Minorities/ Culture Active learning Research 9 13 19 36 77 6 8 17 18 9 9 29 25 61 60 15 13 13 19 60 11 11 6 8 8 7 16 9 41 35 Visualization Project Games Distance Learning /Flipped Class /MOOC Percentage in the conference 6 8 6 10 5 8 6 8 4 7 7 9 29 28 27 < 2 3 8 10 23 63% 80% 69% 85% 75%

Keywords that address pedagogy

Main insights with respect to pedagogical aspects A significant increase is noted in aspects related directly to teaching practices, in particular, assessment, active learning, and collaborative learning. Teaching Methods received the highest attention. Focus is placed on collaborative learning, teamwork, and diversity. A consistent increase is evident in aspects related to distance learning.

Summary 1: Categorization Teaching methods received the most attention Curricula coming in second, at a substantial distance

Summary 2: Keywords Curricula CS1 followed by, at a significant gap, CS0 and K-12 following by computational thinking and interdisciplinary studies Topics Frequencies did not reveal meaningful results Pedagogy Assessment, active learning, and team collaboration.

Summary Total N=477 45% 27% 15% Content categories Keywords categories % of papers that addresses the categories Total N=477 Teaching Methods Curricula CSE Research Recruitment and Retention CS Educators Type of curricula 44% CS topics 20% 8% Pedagogical aspects 75% 5% Maybe all us about CS Educators?

Summary Consistency in interest VS. changes in the field CS educators are interested in Teaching Methods but pay less attention to the educators professional development. This become crucial looking at CS to all / Computational thinking were various new populations of teachers will have to cope the CS content by themselves. In the future: Automatic tools for retrieving this data analysis.

What Are Computer Science Educators Interested In? The Case of ISSEP Conferences? My first and intuitive reflection in relation to the current conf.: A lot of attention regards teachers. Express interest in the field of AI

Thank you noarag@beitberl.ac.il oritha@technion.ac.il