Sensordatafusion Egils Sviestins SaabTech Systems 1
Fusion levels (JDL model) Sources Level 1 Objects Level 2 Situations Level 3 Intentions Level 4 Process
Terminologi Objekt Situationer Avsikter Informationsfusion Sensordata Sensordatafusion Andra data Styrning Styrning Optimering Optimering 3
Mätningar/information räcker inte Modeller krävs! 1 Modeller Matematiska: 2 d x exempel = g + w( t) (kastparab el med bruspåverkan) 2 dt Idéer om verkligheten/ mentala modeller Begränsat av naturlagar, ekonomiska lagar, mänsklig förmåga etc. Mätningar/information snävar in möjligheterna 2 3 4
Från verkligheten... Rån = stöld e.d. som utförs under hot om våld 5
Context
Data processing: Improvement or Destruction? Raw information Sensor User Meaningful information
Synkanalen (hypotetiskt!) Erfarenhet Linjer Ytor Extraktion av kroppar Tolkning Pixels Linjer, ytor Fysiska kroppar Begrepp 8
Hörselkanalen (hypotetiskt!) Erfarenhet Toner Transienter Sortering Tolkning 410Hz, 63 db f k aaa Mänsklig röst Kaffe Frekvens Amplitud Toner Transienter Ljudkällor Ord mm. 9
Early fusion...... or late? WSC 10
Seeing (hypothetical) Pixels Pixels Lines Surfaces Lines Surfaces Physical bodies Known objects WSC 11
Artskilda sensorer Kaffedags Kaffekask?? Kaffetax 12
Tidig fusion - för och emot Mindre risk för tvetydigheter Osäkerheter kan lättare beskrivas statistiskt - Bayes teori kan användas Mindre robust m a p systematiska fel Svårt hantera artskilda källor 13
14 Inte så enkelt...
15 Fusionsprincip i hjärnan?
The Radar Data Processing Chain Receiver Extractor Tracker A12 A07 Raw video Plots (R,az) Tracks (#,x,y,v x,v y,...) WSC 16
Steps in Tracking
The Tracking Cycle Initiation Association Updating WSC 18 Measurements Termination Prediction
Filtering techniques Linear regression (least squares batch processing) (hardly used in this context) (70 s) Alpha-Beta (80 s) Adaptive Kalman (90 s) Interactive Multiple Model (IMM) (2000 s?) Non-linear filtering?
x Linear regression How to handle maneuvering targets??? t
~ ~ x, x & predicted position, speed x$, x& $ updated position, speed x m measured position Prediction step x~ = x$ + x& $ T ~ x& = x& $ Updating step ( ) x$ = x~ + α x ~ m x & $ ~ x x~ m x = x& + β T Alpha-Beta filtering $, & $ x x α and β are tuning constants between 0 and 1 ~, & ~ x x x m new x$, x& $ if α=β=0: Measurement has no effect α=β=1: History has no effect α = 0. 5and β = 0. 5
Current state & uncertainties + Measurement & uncertainties = New state & uncertainties Kalman filtering Like a-b-filter, but: Automatically optimizes a and b Best weighting between history and measurement Output includes estimated accuracy
ẋ Probability densities Update Prediction Measurement x
IMM States State Vector and Filter Type Dynamics Uniform Horizontal Motion ( x,,, &, &, & 1 x2 x3 x1 x2 x3) Linear Kalman filter && x = 0; x& u = 0 u = vertical unit vector Speed Changes ( x,,, &, &, & 1 x2 x3 x1 x2 x3) Linear Kalman filter && x u and && x l = white noise l = longitudinal unit vector && x ( u x& ) = 0 Slow Turns ( x, x, x, x&, x&, x&, ω) 2 1 2 3 1 2 3 nd order Extended Kalman &ω = 0 ω = turn rate Fast Maneuvers ( x,,, &, &, & 1 x2 x3 x1 x2 x3) Linear Kalman filter x&& l = white noise x&& s, x&& t = white noise l s = l t = s t = 0
IMM structure UH SC ST FM Input UH UH UH UH SC SC SC SC ST ST ST ST FM FM FM FM Transition UH SC ST FM Merging UH SC ST FM Propagation UH SC ST FM Updating X Averaging & Output
Bayes teori Observation z p(h ) 1 p(h ) 2 p(h ) 3 p'(h i) p(z H )p(h ) i i p'(h ) 1 p'(h ) 2 p'(h ) 3 26
Associering M målspår, N plottar: hur koppla samman? OBS! Falska/saknade plottar, falska/saknade målspår Närmaste granne? Närmaste granne i statistiskt avstånd? Global optimering statistiskt avstånd (minimera d )? 2 Söka globalt mest sannolika koppling? Hur man än gör kan det bli fel. Motiverar multihypotes 27
Measurement-to-track association Clusters with M measurements and N tracks Form hypotheses like H( z x, z, z x, x ) 1 3 2 3 2 1 Calculate probabilities for each hypothesis, e.g. ( ) ( ) P p z x p P p z x ( P ) d 1 3 s d 3 2 1 d
LPQ association: Plot & Track clusters Track predicted position and search bin * Plot * * Cluster with 3 plots and 2 tracks * * * * * * * * OH103
Bayesian track initiation Given a tentative track. Two hypotheses: H 0 : Track is false H 1 : Track is genuine C n =p(h 1 ): Credibility at scan n Obtained measurement z. Spurious plot density p s. p( H z) = C 1 n+ 1 ( ) p( H ) p z H 1 1 ( ) ( 1 ) [ ] = P p z x + P p C d d s n ( ) ( ) p( H z) p z H p H 0 0 0 ( 1 C ) = p s n
Initiation by Credibility Required: Fast initiation and low false track rate Sequential hypothesis testing Credibility C» likelihood that a potential track is genuine 1 C 0 1 2 3 4 5 6 7 8 Scan # ed
Bildalstrande TV Andra sensorer FLIR (Forward Looking Infrared) Millimetervågsradar SAR (Synthetic Aperture Radar) Icke bildalstrande Störbäringsavtagare Signalspaning IRST (Infrared Search & Track) Akustiska/Hydroakustiska sensorer GPS 32
Decentralized Multi-Radar Tracking Plots Tracking Track correlation & merging System tracks Plots Tracking
Centralized Multi-Radar Tracking Plots Multi-radar tracking System tracks Plots
Filling coverage gaps 1. 2. Two radars Coverage gap Red single radar track lost and reinitiated 3. 4. Decentralized MRT may give confusing picture Centralized MRT performs well
Disadvantages of centralized multi-radar tracking More sensitive to bias errors Bias compensation required Difficult to distribute CPU load on several processors But not impossible Existing data links often do not supply plot level data Sometimes requires hybrid solutions Sensors sometimes include extensive processing Sometimes requires hybrid solutions
Strobes only 150 km
Crossings
Reasons for Multi-Sensor Tracking Radars can be jammed Protective need to keep radars silent Radars don t always give best target detection May support target identification
Based on Target Type Identification Direct observations ESM / IRST measurements Kinematics Each track carries a vector with probabilities of possible target types. Requires a library of target type characteristics
MST+ scenario
Example Lockheed F16 Mirage 2000 Lockheed U2 MiG-25 MiG-29 T 1 T 2 T 3 T 4 T 5 M 1 M 2 M 3 M 4 M 5 M 6 M 7 T 1 T 2 T 3 T 4 T 5 42 3 3 3 1 1 3 3 3 3 1 3 3 3 2 2 3 3 3 3 2 3 3 3 4 5 3 3 3 4 5 3 3 3 4 5 6 3 3 3 4 3 3 3 4 5 6 7 6 6 7
Max altitude Kinematic typing Offline: Create Target Type Database Min/Max speed as function of altitude Max climb rate as function of altitude Max distance from base Max linear/turn acceleration as function of altitude
Step 1 - Collect flight data Max altitude Min/max velocity as function of altitude Max climb rate Max distance from base <Max linear/turn acceleration as function of altitude> Utilise meteorological data if available
Step 2 - Update Probability Vector Collected Flight Data Previous Probability Vector [p(f16),...] Target Type Database Bayes Rule New Probability Vector [p (F16),...]
Avrundning Sensordatafusion - uppgifter om enskilda objekt baserat (mest) på sensordata Bygger oftast på matematiska modeller och Bayesiansk hypotesprövning Många svåra områden återstår Sensorer som ger knepiga data Svårtolkade scenarier (t ex mark och undervatten) Gemensam lägesbild (distribuerad fusion) Fusion av starkt artskilda sensorer Integration med infofusion 46