Crafting Canonical Goal Vectors: Nouns for Goals
Using analogies in a Word2vec embedding space, this study explores how nouns correlate with actions to achieve specific objectives, such as making a salad. By establishing average vectors through various objects, the research delves into the fundamental association between nouns and goals.
Uploaded on Mar 09, 2025 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Informing Action Primitives Through Free-Form Text Nancy Fulda, Ben Murdoch, Daniel Ricks, David Wingate PCCL
Goal: Explain to a Robot How it Can Interact With and Explore the World Where to start? Light fixtures, basket, stove, chairs, window, bottles, lettuce, eggplant, tree, vegetables, bowls, plates, faucet, cupboard, counter top, walls, potted plant, coffee machine, coffee pot?
Goal: Explain to a Robot How it Can Interact With and Explore the World Human goal: Please make me a salad Method: 1: Find relevant objects 2: Find relevant actions
Crafting a Canonical Goal Vector Using Analogies Word2vec embedding space Nouns for goals unlock door :: key carry water :: bottle wash floor :: sponge catch butterfly :: net reach roof :: staircase take bath :: bathtub make money :: job Make a salad :: ?
Crafting a Canonical Goal Vector Using Analogies Nouns for goals Average Vector New Goal (eg: Make a salad ) unlock door :: key carry water :: bottle wash floor :: sponge catch butterfly :: net reach roof :: staircase take bath :: bathtub make money :: job + Noun Space = Make a salad :: ?
Surrounded by a World Full of Objects Goal: make me a salad Where to start? Light fixtures, basket, stove, chairs, window, bottles, lettuce, eggplant, tree, vegetables, bowls, plates, faucet, cupboard, counter top, walls, potted plant, coffee machine, coffee pot?
Finding Relationships Through Cosine Similarity 'eggplant' 0.51116574 'bottles' 0.38117877 'lettuce' 0.47700563 'basket' 0.37218273 'vegetables' 0.44281861 'cupboard' 0.3715933 'plates' 0.41985905 0.36732712 faucet 'stove' 0.40897045 'light fixtures' 0.14254177 'bowls' 0.38328889 'counter top' 0.094668075
Human Guided Exploration Game Human-Defined Goals Zork1 Zork2 Zork3 Candy Omniquest Bunny Detective Mansion Spirit Zenon Cavetrip parc 'enter house', 'get stuff', 'create light', 'move furniture', 'climb tree', 'unlock locks' 'enter buildings', 'get stuff', 'create light', 'move things' 'enter buildings', 'get stuff', 'create light', 'move things' 'get candy', 'search for candy' 'get stuff', 'climb tree', 'wear clothing', 'move things', 'dig' 'enter holes', 'get stuff', 'move things', 'open things', 'burn monsters', 'unlock locks' 'get stuff', 'enter buildings' 'unlock locks', 'take stuff', 'turn on', 'turn off' 'open things', 'get scrolls' 'get stuff', 'look under bed', 'unlock locks', 'turn off light' 'open furniture', 'search furniture', 'get clothes', 'get food', 'get batteries' 'enter buildings', 'get stuff', 'close curtains'
Results Game Score Random Affordance-Based Human Agent Agent Guidance Agent
Thank You! Ben Murdoch murdoch@byu.edu PCCL
