Innovative Focus: Robust Visual Codes for Diverse Captures

Focus: Robust Visual Codes for Everyone Focus: Robust Visual Codes for Everyone Frederik Hermans, Liam McNamara, G bor S r s 1

Visual Codes Too far away Blur Frame Mixing Challenge: Challenge: how to design visual codes that are robust to a wide range of capture conditions? 2

Contributions We propose Focus Suitable for a wide range Partially Partially decodable Significantly improves Focus a new visual code wide range of capture conditions improves on the state of the art Focus addresses the mentioned challenge by encoding data in orthogonal sub orthogonal sub- -channels channels in the frequency domain frequency domain 3

2D Fourier Transform + + + + + + = +... + + = S Any image can be represented as sum of 2D sinusoids An image s spectrum spectrum is a 2D complex matrx sum of 2D sinusoids 4

Encoding a Focus Code Payload Payload Data Chunk 1 ?0,?1,?2,?3, Data Chunk 2 Data Chunk 3 Data Chunk 4 Spectrum Spectrum (2D complex matrix) ?3 ?0 ?2 ?1 ?2 ?0 ?3 ?1 5

Encoding a Focus Code Payload Payload Data Chunk 1 ?0,?1,?2,?3, Data Chunk 2 Data Chunk 3 Data Chunk 4 Focus code Focus code Spectrum Spectrum (2D complex matrix) Inverse FFT 6

Encoding a Focus Code Payload Payload Data Chunk 1 ?0,?1,?2,?3, Data Chunk 2 Data Chunk 3 Data Chunk 4 Focus code Focus code Spectrum Spectrum (2D complex matrix) Inverse FFT Focus codes use as little spatial detail little spatial detail as possible. 7

Decoding a Focus Code Spectrum Spectrum Captured code Captured code FFT Received Payload Received Payload Data Chunk 1 Data Chunk 2 Data Chunk 3 Data Chunk 4 8

Decoding a Focus Code Spectrum Spectrum Captured code Captured code FFT Received Payload Received Payload Data Chunk 1 Data Chunk 2 Data Chunk 3 9

Decoding a Focus Code Spectrum Spectrum Captured code Captured code FFT Received Payload Received Payload Data Chunk 1 10

Decoding a Focus Code Spectrum Spectrum Captured code Captured code FFT Received Payload Received Payload Focus codes can be partially decoded decodable data scales with the reader s sampling rate. scales with the reader s sampling rate. partially decoded. The amount of Data Chunk 1 11

Fountain coding Generate a potentially limitless potentially limitless sequence of symbols from a given source of symbols Recover original k symbols from any k symbols with high probability probability, where k = k + high Data 3 Data 4 Data 1 Data 2 Data 4 Data 2 Data 1 Data 6 Data 7 Data 3 Data 5 Data 2 Data 6 Data 7 Data 3 Data 5 Data 3 Data 4 Data 1 Data 2 12

Experimental Results Evaluate impact of distance Code capacity 1536 bytes Distance 1m ~ 6m Readers Galaxy S6 (2015), Nexus One (2010), Google Glass (2014) distance and camera quality camera quality 13

Impact of Camera Resolution 14

Impact of Distance and Camera Goodput increases increases with decreasing distance distance. 15

Focus & Strata Strata Strata also provides partial decoding and aims to be robust Encodes data in blocks of pixels of varying size blocks of pixels of varying size Focus codes have a lower bit error rate spatial detail spatial detail to represent the payload. lower bit error rate because they use less less 16

Focus & PixNet PixNet PixNet also encodes data in frequency domain; targeted towards high high- -quality cameras quality cameras. Errors in Focus codes concentrated towards end of payload Significantly improved throughput on smart devices throughput on smart devices 17

Conclusions Focus codes represent data in the frequency domain They are readable over worse channels readable over worse channels than existing codes With poorer cameras poorer cameras and over longer distances Can be partially decoded partially decoded if channels is not good enough More: Fountain coding, multi-rate codes... frequency domain longer distances 18

Thank you! Q & A 19