Educational Networking Framework Full Layer Implementation Testing

KENSv2: An Educational Networking Framework for Full Layer Implementation and Testing Keunhong Lee, Joongi Kim, Sue Moon Department of Computer Science, KAIST {khlee, joongi}@an.kaist.ac.kr, sbmoon@kaist.edu

Networking and Communication - From ACM Computer Science Curricula 2013 Principles and practice interact. Networking is real and many of the design choices that involve networks also depend on practical constraints. Students should be exposed to these practical constraints by experimenting with networking, using tools, and writing networked software. 2

To Learn Is to Do Computer Networks I, Case Western Reserve University 6 written homework assignments; 2 course projects (each project takes about one month); midterm exam (about 1 hour); final exam (3 hours). CS144: Introduction to Computer Networking, Stanford University Student grades are based on a combination of two exams, four programming assignments, two problem sets, a technical writing assignment, and online quizzes embedded in the class videos. Computer Networks, Williams College Students complete weekly written assignments, one significant programming project, and take two open-book takehome examinations. CS2200: Introduction to Systems and Networking, Georgia Institute of Technology Two midterms, one final, 5 homeworks, 5 projects (two architecture projects: processor datapath and control implementation, and augmenting processor to handle interrupts; three OS projects: paged virtual memory management, multithreaded processor scheduler using pthreads, and reliable transport layer implementation using pthreads). Plus an extra- credit project (a uniprocessor cache simulator) 3

Assignments in Network Courses Protocol implementations TCP, UDP, FTP, HTTP Algorithm implementations Distance vector Congestion control Network configuration/operation/monitoring Wireshark Mininet + OpenFlow BGP, NAT, IPsec 4

Assignments in Network Courses Protocol specification TCP, UDP, FTP, HTTP Algorithm implementation Distance vector Congestion control Network configuration/operation/monitoring Wireshark Mininet + OpenFlow BGP, NAT, IPsec We want to build an educational framework for building TCP 5

What We Have Done at KAIST Pintos (SIGCSE 09) CS330: Operating Systems and Lab Building a toy operating system STCP (from COS461 at Princeton Univ.) EE323: Computer Networks Building lightweight version of TCP (protocol only) KENSv1 (http://nclab.kaist.ac.kr/kens/) CS441: Introduction to Computer Networking Building students own TCP layer Taught since 2005~ 6

Feedback on KENSv1 since 2005 7

Oh, what a silly bug! We will fix it for the sake of next year students. -TA 8

Summary of the Feedback Hard to understand framework code which is already implemented Put handler callbacks instead of filling missing code parts Grading projects by looking into students code is time-consuming (TA) Grade based on the test suite output I don t know whether I m doing correctly during the project Allow partial development and partial testing 9

Requirements for an Educational TCP Framework Cover core concepts of TCP 3-way handshaking Packet recovery/reordering Congestion control Easy to follow for students Reduce burden of handling OS functionalities Step-by-step implementation Easy to grade for TAs Automated testing without reading students code 10

What makes building an educational framework for network systems challenging? 11

Example: 3-Way Handshaking Automated test suit Assignment syscall: accept {} syscall: connect {} 12

A Network System Needs a Counterpart Automated test suit Assignment Server Client syscall: accept ? {Complete} syscall: connect {} 13

Both Server/Client Should Be Done At Once Automated test suit Assignment Server syscall: accept Test logic for accept() {Complete} Fails because the client is not done Client syscall: connect {} 14

Test Suite Should Contain TCP Logic Itself Automated test suit Assignment Server Client syscall: accept Test logic for accept() connect() logic as the counterpart {Complete} syscall: connect {} 15

Test Code Copied In Assignment Automated test suit Assignment Server Client syscall: accept Test logic for accept() connect() logic as the counterpart {Complete} syscall: connect connect() logic copied into the assignment {} 16

Test Code Is Also a Good Teaching Material Automated test suit Assignment Server Client (binary form) syscall: accept Blackbox {Complete} No idea what is going on w/ the test syscall: connect {} 17

An Adversary In Binary Form Solves The Problem Automated test suit Assignment Server Test (open source) syscall: accept Test logic for accept() connect() is called during the test {Complete} Client (binary form) syscall: connect Adversary Solution {} 18

KENSv2 KAIST Educational Networking System v2 19

KENSv2 Framework 20

KENSv2 Framework Behavior of virtual applications Partial testing for coupled functionalities Students main assignment IP is not handled in our lecture We provide solution binary Total ordering of network events for each testing situation Offload protocol verification into Wireshark 21

KENSv2 includes Virtual Host Event Generator Captured packets can be analyzed by external packet analysis tools Test suites Test cases for CUnit, an open source framework for unit testing Template source 22

KENS Assignments Add Up to One s Own TCP Driver struct kens_tcp_driver_t { void (*startup)(kens_system_lib*); void (*shutdown)(kens_tcp_driver*); my_context (*open)(kens_tcp_driver*, int*); void (*close)(kens_tcp_driver*, my_context, int*); bool (*bind)(kens_tcp_driver*, my_context, const struct sockaddr *, socklen_t, int*); bool (*listen)(kens_tcp_driver*, my_context, int, int*); bool (*connect)(kens_tcp_driver*, my_context, const struct sockaddr *, socklen_t, int*); bool (*accept)(kens_tcp_driver*, my_context, int*); bool (*getsockname)(kens_tcp_driver*, my_context, struct sockaddr *, socklen_t *, int*); bool (*getpeername)(kens_tcp_driver*, my_context, struct sockaddr *, socklen_t *, int*); void (*timer)(kens_tcp_driver*, my_context, int); void (*ip_dispatch_tcp)(kens_tcp_driver*, struct in_addr, struct in_addr, const void *, size_t); int (*app_dispatch_tcp)(kens_tcp_driver*, my_context, const void*, size_t); }; 1) 2) 3) 4) 5) Initializer Implement 6) 7) System call handler all 13 callback functions! 8) 9) 10) Interact with the system, Packet I/O 11) 12) 13) 23

Benefits of KENSv2 System call and context management Explicit ordering of network events Unit tests for partial testing and grading Use of other tools for protocol specification verification and algorithm correctness check 24

Benefits of KENSv2 (1) System call and context management Do not have to handle with complex OS features Explicit ordering of network events Unit tests for partial testing or grading Use of other tools for protocol specification verification and algorithm correctness check 25

File Descriptors Are Handled by KENS System Call Layer 26

System Calls Are Mapped to Corresponding Handlers Linux KENS bind(fd, args ) kbind(fd, args ) invoke bind( ) invoke kbind( ) interrupt 0x80 KENS syscall layer ret sys_bind(fd, args ) my_bind(ctx, args , &ret) ret bind( ) ret kbind( ) Kernel space User space Kernel space User space 27

Benefits of KENSv2 (2) System call and context management Explicit ordering of network events Tests are easy to understand and reproduce Unit tests for partial testing or grading Use of other tools for protocol specification verification and algorithm correctness check 28

Ordering Of Network Events server() { server_socket = socket(); bind(server_socket, address); listen(server_socket); client_socket = accept(server_socket); send(client_socket, data); close(client_socket); close(server_socket); } client() { client_socket = socket(); connect(client_socket, address); receive(client_socket, data); close(client_socket); } Network events can be interleaved in various way 29

Example: close() Test_Close_1() { ... server->close(); flush_packets(); client->close(); flush_packets(); } 30

Example: close() Test_Close_2() { ... server->close(); client->close(); flush_packets(); } 31

Benefits of KENSv2 (3) System call and context management Explicit ordering of network events Unit tests for partial testing and grading Summary report for each testing/grading criteria Use of other tools for protocol specification verification and algorithm correctness check 32

Example: Test Result (success) Suite: testListen Test: __testListen_Accept_Before_Connect ...passed Test: __testListen_Accept_After_Connect ...passed Test: __testListen_Accept_Multiple ...passed Test: __testListen_Multiple_Interfaces ...passed 33

Example: Test Result (failed) Suite: testBind Test: __testBind_Simple ...passed Test: __testBind_GetSockName ...passed Test: __testBind_DoubleBind ...passed Test: __testBind_OverlapPort ...passed Test: __testBind_OverlapClosed ...passed Test: __testBind_DifferentIP_SamePort ...FAILED 1. testbind.c:244 - CU_ASSERT_EQUAL(err,0) 2. testbind.c:245 - CU_ASSERT_TRUE(ret) Test: __testBind_SameIP_DifferentPort ...passed 34

Benefits of KENSv2 (4) System call and context management Explicit ordering of network events Unit tests for partial testing or grading Use of other tools for protocol specification verification and algorithm correctness check Offload complex analysis to existing solutions 35

Wireshark Wireshark labs in Has basic stateless verification such as checksum, option flags, field range, etc. Also contains context-aware analysis like SEQ/ACK analysis to detect unseen ACKs. 36

Pcap Compatible Logging KENSv2 implements virtual IP layer Dumping packets in forms of pcap enables analysis with Wireshark (or other useful packet analysis tools) Wireshark does packet verification! 37

Network I/O - Without Drop 38

Network I/O - With Drop 39

Bytes-on-flight (SEQ/ACK Analysis From Wireshark) 40

KENSv2 Assignment #1 Introduction to KENS framework Implement basic system call functionality open(), close(), getsockaddr() Maintain opened sockets Check address collision during bind() 41

KENSv2 Assignment #2 Implement 3-way handshaking listen(), accept(), connect() Not transmitting actual data stream 42

KENSv2 Assignment #3 Data transfer over a reliable network write(), read() SEQ/ACK manipulation 4-way handshaking close() After this assignment, all test suites should run over a reliable network 43

KENSv2 Assignment #4 Connection setup and data transfer over an unreliable network Reordering Retransmission Timeouts (e.g. TIME_WAIT of TCP state diagram) Same test suite of assignment #3 should pass over an unreliable network (tested several times?????) 44

KENSv2 Assignment #5 AIMD congestion control algorithm RTT estimate Window size management Fast retransmission Slow start Run the same test suite from assignment #3 TAs check window size change with Wireshark 45

KENSv3? POSIX compatible system calls No more kbind, klisten, etc Running multi-threaded application While keeping the total ordering of the events Reproducible multi-threaded application! Unified interface for multiple protocols IPPROTO_TCP, IPPROTO_UDP, and many other improvements 46

KENSv2 Open source available at https://github.com/ANLAB-KAIST/KENSv2 KENSv3 will be released after June 2015 Currently beta-testing in CS341 at KAIST http://an.kaist.ac.kr/courses/2015/cs341 47