Continuous Spaces and Stock Grounding Tasks Explained

Grounding Language with Points and Paths in Continuous Spaces Berkeley Berkeley N L P N L P Jacob Andreas and Dan Klein UC Berkeley

Formal grounding On June 26th, Facebook stock cost $65 per share quote { date: 2014-06-26, stock: FB, price: $65 }

Perceptual grounding On June 26th, Facebook stock rebounded after a bruising swoon ?

Perceptual grounding On June 26th, Facebook stock rebounded after a bruising swoon

Perceptual grounding On June 26th, Facebook stock rebounded after a bruising swoon A after B A, B A before B B, A rebounded { sgn(slope) = +1 } bruising { sgn(slope) = -1, abs(slope) = +2.3 }

Continuous spaces everywhere On June 26th, Facebook stock rebounded after a bruising swoon A deep red sunset Keep a little to the left of the post Beat the eggs gently, until they form stiff peaks

Three tasks Time series Navigation Color

Predicting colors H V S pastel blue dark pastel blue blue

Regression model H V S pastel blue dark pastel blue blue

Regression model dark pastel blue 0 H 0 216 216 + = + 0 S -37 43 80 V -40 -25 75 90

Regression model dark pastel blue

Regression model H 216 S V 75 43 dark pastel blue {dark, pastel, blue}

Experiment setup

Sample predictions pale blue dark brown electric green pale green indigo

Prediction error 0.4 0.3 Baseline 0.2 Last word Full model 0.1 0 Hue Sat Val Mean

A guessing game pale blue

A guessing game 1 0.86 0.81 0.78 0.8 Baseline 0.6 0.50 Last word Full model 0.4 Human 0.2 0 Prediction accuracy

Predicting time series 1 2 stocks rebounded after a bruising swoon 2 1

Predicting time series stocks rebounded after a bruising swoon

Predicting time series sgn(slope): -1 abs(slope): 3.1 curvature: 0.5 sgn(slope): 1 abs(slope): 2.7 curvature: -0.1 2 1 after a bruising swoon {after, a, bruising, swoon} stocks rebounded {stocks, rebounded}

Learning & inference Need parameters for linear prediction model & log-linear alignment model: easy with EM For small number of path segments, possible to sum exactly over latent alignments Otherwise, approximation of your choice

Experiment setup Market rallies to new highs

Sample predictions Reference Predicted U.S. stocks end loweras economic worries persist [U.S. stocks end lower]2 [as economic worries persist]1

A guessing game 0.8 0.72 0.61 0.59 0.6 0.5 Baseline No alignment 0.4 Full model Human 0.2 0 Prediction accuracy

Peeking at parameters sgn(slope) abs(slope) rise 0.27 -0.78 0 swoon -0.57 0.28 sharply -0.22

Following instructions and then we're going to turn north again and immediat-- well a distance below that turning point there's a fenced meadow but you should be avoiding that by quite a distance okay so we've turned and we're going up north again continue straight up north and then we're going to turn to the west on a curvature right sort of

Navigation results 0.6 0.5 0.4 Branavan 0.3 Vogel This work 0.2 0.1 0 Precision Recall F-measure

Conclusions New model for predicting grounded representations of meaning in arbitrary real- valued spaces Beats strong baselines on a diverse range of tasks Code and data available online at http://cs.berkeley.edu/~jda