Integrating Data Streams with Probabilities and Relational Databases Using Bayesian Networks

The Integration of Data Streams with Probabilities and Relational Database using Bayesian Networks Ryo Sato Hideyuki Kawashima Hiroyuki Kitagawa University of Tsukuba SeNTIE 2008

Outline Background and Purpose Proposal Continuous reasoning of uncertain situations Integration of reasoning results and relational data Conclusions and Future Work

Various Sensor Data Streams RFID GPS Camera Wireless Sensor Nodes Mote, SunSPOT

Data Streams Processing Example of applications Monitoring for fire detection Fire detection (Application) SPE stream processing engine The SPE processes data streams. Continuous query processing Integration with other relations STREAM(Stanford), Borealis(MIT) Data streams processing engine ID Temp Time 1 25 12:00

Concrete example : Fire detection in power plant Monitor data streams generated by sensor devices attached to a gas turbine in the power plant Does temperature sensor indicate an abnormal value ? Where is setting place of the sensors which indicate an abnormal value ? Register query Monitoring query SPE Query result DB Senor data of gas turbine Temperature Number of revolutions Power plant

ProblemAn uncertain situation When a gas turbine temperature is abnormal Fire or Error of sensor device The user should reason an uncertain situation such as Fire or Error of sensor device (Existence SPE can't correspond) If SPE supports such an uncertain situation The user is released from the reasoning processing. Query example Tell me which places may be in fire ?

Purpose and Approach Purpose Construct an SPE which can express uncertain situations The SPE processes probabilistic data streams Approach Expression of uncertain situations Reason situation using Bayesian Networks Integration of reasoning results and relational data Integration of probabilistic table and relational table

Outline Background and Purpose Proposal Continuous reasoning of uncertain situations Integration of reasoning results and relational data Conclusions and Future Work

Continuous reasoning of uncertain situations It is useful to reason an uncertain situation such as error of sensor device or a fire" at the gas turbine high temperature, and to obtain a probabilistic value of events. Bayesian Networks Typical probability reasoning technique to express an uncertain situation Example Spam filter etc. An uncertain situation is reasoned using Bayesian Networks

Bayesian Networks (BN) Each node is connected based on causal relationships. Each node has conditional probability table(CPT) One node is treated as one event (high temperature and Fire) Spread probabilities using CPT when event occurrences are detected. The probability value of each event changes true false true false Detect high temp by temperature sensor devices P(High Temp) 1.0 0.0 P(High Temp) High Temp 0.1 0.9 true false True False P(Error|High Temp=true) 0.7 0.3 P(Fire|High Temp=true) 0.8 0.2 P(Fire|High Temp=false) 0.1 0.9 P(Error|High Temp=false) 0.2 0.8 true false true false Calculate fire probability Calculate error probability P(Error) 0.7 0.3 P(Fire) 0.8 0.2 Error Fire true false true false P(Error) 0.21 0.79 P(Fire) 0.17 0.83

Extension of BNContinuous Event The model of BN is static It cannot deal with unbounded event stream Probability update for each event occurrence The occurrence of more than two events cannot be expressed at the same time. After the event occurs, the occurrence duration cannot be expressed. Proposition: Lifespan The lifespan is the duration of event occurrence. Lifespan is set for every event. The length of the arrow shows Lifespan When it executes a query, it defines occurring event based on lifespan Execute probabilistic calculation based on occurring event High Temp Two events are detected in room 303 No events are detected High Temp in room 303 event is detected in room 303 Person exists in room 303 Time T1 T3 T2

Outline Background and Purpose Proposal Continuous reasoning of uncertain situations Integration of reasoning results and relational data Conclusions and Future Work

Integration of reasoning results and relational data (outline) Event detector Tell me which places may be in fire ? Table Name Monitoring T Event High Temp Room Guest Name Pr Room 101 T1 0.7 303 John Ev Pr Room Guest Name BN-Window operator Room Fire The query is executed at time T4. 0.7 High temp in 101 303 303 RoomInfo John Table generation operator Table Name Monitoring operator BN-Map T Event High Temp Room Person exists in 202 101 T1 Room BN-Obj High Temp High temp in 303 Person exists High Temp Pr 1.0 202 Detection T2 101 303 Monitoring T3 Update Pr Fire Update Pr Error 303 T0 BN-Buffer Monitoring T1 T2 T3 T4 T5

Table generate operator(1/2): BN is stored in the table Expression of BN in table Problem Since BN has a graphical model, it cannot express in relational data model Use object relational data model BN expresses BN-Obj using abstract data type(ADT) Room is room-number that BN performs a situation estimate BN-Obj that is the object type of BN is stored in the BN-Obj attribute. Bayesian Networks Room BN-Obj High Temp 202 BN-Obj1 303 BN-Obj2 Error Fire 303 BN-Obj3 Problem it can store BN, but it cannot integrate

Table generate operator(2/2): The table extraction method Introduce methods to generate table (Room, Ev (event name), Pr ) BN-Obj type Node type getNode(Id or Pr or Ev, Sign, Value) Convert into the node type satisfied the condition from BN-Obj type Search for the condition at Id, Ev and Pr. ( Ev = Fire , Pr >= 0.8) Node type Node type child(), descendant(), parent(), ancestor(), sibling() Convert into the node type such as child or parent of the node appointed in getNode method Node type Relation makeTuple() Convert into a table including a probability of (Room,Ev,Pr) from node type Example: getNode(Ev= High Temp ). child(). makeTuple() High Temp Room Ev Pr 303 Error 0.2 Relational table 303 Fire 0.1 Error Fire

Integration of reasoning results and relational data (outline) Event detector Tell me which places may be in fire ? Guest Name Pr Room Table Name Monitoring T Event High Temp Room 0.7 3B303 John 101 T1 Guest Name BN-Window operator Room Ev Pr Room 303 RoomInfo Fire 0.7 John 303 BN-Map Table Name Monitoring operator Table generation operator T Event High Temp Room 101 T1 High Temp Room BN-Obj Person exists High Temp Pr 1.0 202 Detection T2 101 303 Monitoring T3 Fire Update Pr Update Pr Error 303 BN-Buffer Monitoring

Integration of probabilistic table Probabilistic table Table where probability value is given to each tuple Each tuple expresses an independent event Problem when integration with probabilistic table and relational table In the integration of the probability tables, the product of the probability value of each tuple is a probability value after the integration The integration of probability table and relational table without probability values is impossible. Set probability value 1.0 (100%) to a relational table and enable integration Set probability value 1.0 Room Ev Pr Room Guest Name Pr 303 Fire 0.4 303 John 1.0 Probabilistic table Relational table Room Ev Room Guest Name Pr 303 Fire 303 John 0.4 (1.0 0.4)

Integration of reasoning results and relational data (outline) Event detector Tell me which places may be in fire ? Guest Name Pr Room Table Name Monitoring T Event High Temp Room 0.7 3B303 John 101 T1 Guest Name BN-Window operator Room Ev Pr Room 303 RoomInfo Fire 0.7 John 303 BN-Map Table Name Monitoring operator Table generation operator T Event High Temp Room 101 T1 High Temp Room BN-Obj Person exists High Temp Pr 1.0 202 Detection T2 101 303 Monitoring T3 Fire Update Pr Update Pr Error 303 BN-Buffer Monitoring

Query Language (1/2) Query language is based on SQL. Extension Produce relational table from BN Example I demand the fire probability value in room 303 every 10 seconds. MASTER 10sec SELECT FROM WHERE bn.getNode(Ev= Fire ).makeTuple() tableBN tableBN.Room = 303 High Temp Room bn(BN-Obj) tableBN Room Ev Pr 202 BN-Obj 303 Fire 0.4 303 BN-Obj Error Fire

Query Language (2/2) Integration example of relational table and probabilistic table I demand the fire probability and room information every 10 seconds in room 303 MASTER SELECT FROM WHERE 10sec * tableR, (SELECT FROM WHERE bn.getNode(Ev= Fire ).makeTuple() tableBN tableBN.Room = 303 ) as bnpr tableR.Room = bnpr.Room Room Guest Name Phone Room Ev Pr tableBN 303 John 6010 303 Fire 0.4 Room bn(BN-Obj) tableR bnpr Ev1 202 Room Guest Name Phone Room Ev Pr Ev2 Ev3 303 John 6010 303 Fire 0.4 303 High Temp 403 Mike 8120 Error Fire

Outline Background and Purpose Proposal Continuous reasoning of uncertain situations Integration of reasoning results and relational data Conclusions and Future Work

Conclusions and Future Work Conclusions Proposal of probabilistic data streams model Continuous reasoning process using Bayesian Networks Integration of Bayesian Networks and RDB Integration of probabilistic table and relational table Future work Implement probabilistic ORDBMS Use BN in the real environment (44 cameras room)