Sensordatafusion. Egils Sviestins SaabTech Systems

Transkript

1 Sensordatafusion Egils Sviestins SaabTech Systems 1

2 Fusion levels (JDL model) Sources Level 1 Objects Level 2 Situations Level 3 Intentions Level 4 Process

3 Terminologi Objekt Situationer Avsikter Informationsfusion Sensordata Sensordatafusion Andra data Styrning Styrning Optimering Optimering 3

4 Mätningar/information räcker inte Modeller krävs! 1 Modeller Matematiska: 2 d x exempel = g + w( t) (kastparab el med bruspåverkan) 2 dt Idéer om verkligheten/ mentala modeller Begränsat av naturlagar, ekonomiska lagar, mänsklig förmåga etc. Mätningar/information snävar in möjligheterna 2 3 4

5 Från verkligheten... Rån = stöld e.d. som utförs under hot om våld 5

6 Context

7 Data processing: Improvement or Destruction? Raw information Sensor User Meaningful information

8 Synkanalen (hypotetiskt!) Erfarenhet Linjer Ytor Extraktion av kroppar Tolkning Pixels Linjer, ytor Fysiska kroppar Begrepp 8

9 Hörselkanalen (hypotetiskt!) Erfarenhet Toner Transienter Sortering Tolkning 410Hz, 63 db f k aaa Mänsklig röst Kaffe Frekvens Amplitud Toner Transienter Ljudkällor Ord mm. 9

10 Early fusion or late? WSC 10

11 Seeing (hypothetical) Pixels Pixels Lines Surfaces Lines Surfaces Physical bodies Known objects WSC 11

12 Artskilda sensorer Kaffedags Kaffekask?? Kaffetax 12

13 Tidig fusion - för och emot Mindre risk för tvetydigheter Osäkerheter kan lättare beskrivas statistiskt - Bayes teori kan användas Mindre robust m a p systematiska fel Svårt hantera artskilda källor 13

14 14 Inte så enkelt...

15 15 Fusionsprincip i hjärnan?

16 The Radar Data Processing Chain Receiver Extractor Tracker A12 A07 Raw video Plots (R,az) Tracks (#,x,y,v x,v y,...) WSC 16

17 Steps in Tracking

18 The Tracking Cycle Initiation Association Updating WSC 18 Measurements Termination Prediction

19 Filtering techniques Linear regression (least squares batch processing) (hardly used in this context) (70 s) Alpha-Beta (80 s) Adaptive Kalman (90 s) Interactive Multiple Model (IMM) (2000 s?) Non-linear filtering?

20 x Linear regression How to handle maneuvering targets??? t

21 ~ ~ x, x & predicted position, speed x$, x& $ updated position, speed x m measured position Prediction step x~ = x$ + x& $ T ~ x& = x& $ Updating step ( ) x$ = x~ + α x ~ m x & $ ~ x x~ m x = x& + β T Alpha-Beta filtering $, & $ x x α and β are tuning constants between 0 and 1 ~, & ~ x x x m new x$, x& $ if α=β=0: Measurement has no effect α=β=1: History has no effect α = 0. 5and β = 0. 5

22 Current state & uncertainties + Measurement & uncertainties = New state & uncertainties Kalman filtering Like a-b-filter, but: Automatically optimizes a and b Best weighting between history and measurement Output includes estimated accuracy

23 ẋ Probability densities Update Prediction Measurement x

24 IMM States State Vector and Filter Type Dynamics Uniform Horizontal Motion ( x,,, &, &, & 1 x2 x3 x1 x2 x3) Linear Kalman filter && x = 0; x& u = 0 u = vertical unit vector Speed Changes ( x,,, &, &, & 1 x2 x3 x1 x2 x3) Linear Kalman filter && x u and && x l = white noise l = longitudinal unit vector && x ( u x& ) = 0 Slow Turns ( x, x, x, x&, x&, x&, ω) nd order Extended Kalman &ω = 0 ω = turn rate Fast Maneuvers ( x,,, &, &, & 1 x2 x3 x1 x2 x3) Linear Kalman filter x&& l = white noise x&& s, x&& t = white noise l s = l t = s t = 0

25 IMM structure UH SC ST FM Input UH UH UH UH SC SC SC SC ST ST ST ST FM FM FM FM Transition UH SC ST FM Merging UH SC ST FM Propagation UH SC ST FM Updating X Averaging & Output

26 Bayes teori Observation z p(h ) 1 p(h ) 2 p(h ) 3 p'(h i) p(z H )p(h ) i i p'(h ) 1 p'(h ) 2 p'(h ) 3 26

27 Associering M målspår, N plottar: hur koppla samman? OBS! Falska/saknade plottar, falska/saknade målspår Närmaste granne? Närmaste granne i statistiskt avstånd? Global optimering statistiskt avstånd (minimera d )? 2 Söka globalt mest sannolika koppling? Hur man än gör kan det bli fel. Motiverar multihypotes 27

28 Measurement-to-track association Clusters with M measurements and N tracks Form hypotheses like H( z x, z, z x, x ) Calculate probabilities for each hypothesis, e.g. ( ) ( ) P p z x p P p z x ( P ) d 1 3 s d d

29 LPQ association: Plot & Track clusters Track predicted position and search bin * Plot * * Cluster with 3 plots and 2 tracks * * * * * * * * OH103

30 Bayesian track initiation Given a tentative track. Two hypotheses: H 0 : Track is false H 1 : Track is genuine C n =p(h 1 ): Credibility at scan n Obtained measurement z. Spurious plot density p s. p( H z) = C 1 n+ 1 ( ) p( H ) p z H 1 1 ( ) ( 1 ) [ ] = P p z x + P p C d d s n ( ) ( ) p( H z) p z H p H ( 1 C ) = p s n

31 Initiation by Credibility Required: Fast initiation and low false track rate Sequential hypothesis testing Credibility C» likelihood that a potential track is genuine 1 C Scan # ed

32 Bildalstrande TV Andra sensorer FLIR (Forward Looking Infrared) Millimetervågsradar SAR (Synthetic Aperture Radar) Icke bildalstrande Störbäringsavtagare Signalspaning IRST (Infrared Search & Track) Akustiska/Hydroakustiska sensorer GPS 32

33 Decentralized Multi-Radar Tracking Plots Tracking Track correlation & merging System tracks Plots Tracking

34 Centralized Multi-Radar Tracking Plots Multi-radar tracking System tracks Plots

35 Filling coverage gaps Two radars Coverage gap Red single radar track lost and reinitiated Decentralized MRT may give confusing picture Centralized MRT performs well

36 Disadvantages of centralized multi-radar tracking More sensitive to bias errors Bias compensation required Difficult to distribute CPU load on several processors But not impossible Existing data links often do not supply plot level data Sometimes requires hybrid solutions Sensors sometimes include extensive processing Sometimes requires hybrid solutions

37 Strobes only 150 km

38 Crossings

39 Reasons for Multi-Sensor Tracking Radars can be jammed Protective need to keep radars silent Radars don t always give best target detection May support target identification

40 Based on Target Type Identification Direct observations ESM / IRST measurements Kinematics Each track carries a vector with probabilities of possible target types. Requires a library of target type characteristics

41 MST+ scenario

42 Example Lockheed F16 Mirage 2000 Lockheed U2 MiG-25 MiG-29 T 1 T 2 T 3 T 4 T 5 M 1 M 2 M 3 M 4 M 5 M 6 M 7 T 1 T 2 T 3 T 4 T

43 Max altitude Kinematic typing Offline: Create Target Type Database Min/Max speed as function of altitude Max climb rate as function of altitude Max distance from base Max linear/turn acceleration as function of altitude

44 Step 1 - Collect flight data Max altitude Min/max velocity as function of altitude Max climb rate Max distance from base <Max linear/turn acceleration as function of altitude> Utilise meteorological data if available

45 Step 2 - Update Probability Vector Collected Flight Data Previous Probability Vector [p(f16),...] Target Type Database Bayes Rule New Probability Vector [p (F16),...]

46 Avrundning Sensordatafusion - uppgifter om enskilda objekt baserat (mest) på sensordata Bygger oftast på matematiska modeller och Bayesiansk hypotesprövning Många svåra områden återstår Sensorer som ger knepiga data Svårtolkade scenarier (t ex mark och undervatten) Gemensam lägesbild (distribuerad fusion) Fusion av starkt artskilda sensorer Integration med infofusion 46