Analysera strömmande data med Event Stream Processing SAS Forum Börje Edlund Chief Architect borje.edlund@sas.com Twitter @BorjeEdlund Linkedin: Börje Edlund
SAS EVENT STREAM PROCESSING ENGINE Behovet av Event Stream Processing Användningsexempel Översikt SAS Event Stream Processing Engine och Streaming Analytics Demonstration Copyr i g ht 2013, SAS Ins titut e Inc. All rights res er ve d.
BIG DATA FRÅN STORE IT -> SCORE IT TILL SCORE IT -> STORE IT INTRESSET SKIFTAR TILL STREAMING ANALYTICS FÖR ATT AGERA och FATTA BESLUT I TID Internet Sociala Media Kundinteraktioner Banktransaktioner Telecommunikation Sensorer Intelligenta enheter «Internet of Things».och mycket mer Copyr i g ht 2015, SAS Ins titut e Inc. All rights res er ve d.
ANVÄNDNINGSOMRÅDEN Exempel på områden för ESP Tre korta kundexempel
EXEMPEL PÅ OMRÅDEN ECOMMERCE OPTIMISATION Clickstream Analysis Optimize user experience Real time marketing and advertising CONNECTED DEVICES (IoT) Real time sensors survey RT anomalies detection RT triggering and decision Predictive maintenance DECISION MANAGEMENT Complement SAS EDM RT decisions on event streams Deploy additive and incremental models FRAUD DETECTION Real time transaction analysis User behavior detection Customer profile correlation RT alerts and case management TELECOMUNICATIONS CDRs analysis Real Time Marketing Fraud detection IT systems survey CAPITAL MARKETS Complement SAS HP Risk Reduce time from trading to reporting Continuous calculations
Connected Truck FORTSÄTT RULLA! MÅL Förutsäga underhållsbehov för enskilda lastbilar innan fel uppstår Proaktivt serva lastbilar vid lämplig tidpunkt Erbjuda kunder ny typ av nivå på service plan Process: Data från 60+ sensors/truck. Integrerat med produkt detaljer, garanti händelser, och andra data för context. Analytiska modeller för att förutsäga sannolikheten för specifikt fel inom 30 dagar. Resultat Modeller förutsäger sannolika fel 30 dagar före med 90% tillförlitlighet Förbättrad insikt i vad som orsakar fel leder till ökat produktivitet Image credit: Mike, https://www.flickr.com/people/pmiaki/
UNDVIKA AVBROTT I KRITISKA KOMPONENTER UTMANING Monitorera Elektroniska Sänkbara Pumpars (ESP) effektivitet och riggar djuphavsborrning En trasig pump är $2M/dag; en dags förlorad produktivitet är $20M föresenad intäkt Effekter Millisekundsnabb automatiserad upptäckt av spektrala prestanda och sensorers datakvalitet Analys i dataströmmen för att effektivt undvika problem Löser fler händelsedrivna problem och snabbare än förut Olje & Gasbolag Över 2.1 Millioner sensorer 3 miljard rader data per minut monitoredade för potentiella fel (temperature, vibration,..) prediktiva modeller för att detektera mönster som indikerar fel C opy r i g ht 2 0 1 3, S A S Ins t i t ute Inc. A l l r i g hts r es er v ed.
Europeiskt Telebolag BESLUT OM ERBJUDANDE INOM DELAR AV EN SEKUND När hastigheten spelar roll Så är tid pengar Snabbt urval av intressanta händelser För sen reaktion och reaktionen tappar sitt värde! 10x Förbättring av svar på erbjudande Latency 100-250 ms Events coming in 20,000 requests/sec SAS REAL TIME DECISION MANAGER Relevant events 15 requests/sec ESP Latency 5-15 ms Copyr i g ht 2012, SAS Ins titut e Inc. All rights res er ve d. Image: https://www.flickr.com/photos/spielbrick/
SAS EVENT STREAM PROCESSING ENGINE OCH STREAMING ANALYTICS Hur fungerar det?
VAD KARAKTERISERAR SAS EVENT STREAM PROCESSING PERFORMANCE SAS Event Stream Processing can process huge volumes of streaming data flowing at very high rates (Millions of events/sec) with very low latency (<1 millisec) ENTERPRISE CLASS SAS Event Stream Processing provides seamless and flexible integration with existing IT architecture components in addition to high availability and guaranteed delivery FLEXIBILITY SAS Event Stream Processing provides flexible modeling interfaces, including the SAS Event Stream Processing Studio to enable new analysis & processing models to be quickly developed and modified STREAMING ANALYTICS SAS Event Stream Processing leverages top of class SAS Analytics capabilities including Text Analytics and Data Quality to provides the most accurate and efficient pattern detection
Event Stream Processing (ESP) vs. relational Database Management System (RDBMS) ESP RDBMS ESPs store the queries and continuously stream data through the queries Databases store the data and periodically run queries against the stored data EVENTS INCREMENTAL RESULTS QUERIES RESULTS Copyr i g ht 2012, SAS Ins titut e Inc. All rights res er ve d.
Sas event stream processing LOGICAL MODELING FLOW Processes Data Stores Streaming Events LASR Server SAS-generated Insights Enrichment Analytic Data Rules Business Models
Event Stream Processing Studio
ESP STREAM VIEWER Out-of-the-box event streams visualization for ESP model designers Display output event streams HTML5 Web interface Table and Graphs Streaming and update modes Manual publishing of events
Broker Surveillance POST-TRADE ORDER PRACTICE & COMPLIANCE ALERTING Event Stream Sources/Publishers Trades Market Feed Brokers Restricted Securities Venue Trade Windows Trades Brokers of Interest Restricted Securities Venues Trades Event Stream Processing Server Trades of Large Size (filter) Trades of Interest (join) Frontrunning Patterns (procedural) Restricted Sales (join) Marking Open/Close Patterns (procedural) Broker Alerts (aggregate) Broker Alerts: Broker aggregates for each alert type and total. Front-running: broker buys securities for his own account before buying the same securities for his customer, then sells when the price rises; or broker sells securities out of his personal accounts prior to selling the same securities for his clients. Restricted Sale: sales of securities that have ownership restrictions. Front- Running Alert Restricted Sale Alert Open/Close Mark Alert Broker Alerts Case Management GUI Marking the Close: attempting to influence the closing price of a security by executing purchases at the close of normal trading hours. Marking the Open: attempting to influence the opening price of a security by making trades at the opening of normal trading hours.
SAS EVENT STREAM PROCESSING ENGINE OCH STREAMING ANALYTICS Kort Demonstration
Analysera strömmande data med Event Stream Processing Börje Edlund Chief Architect borje.edlund@sas.com Twitter @BorjeEdlund Linkedin: Börje Edlund SAS Forum Några länkar: The connected truck https://m.youtube.com/watch?v=sku8dmrwipe SAS Event Stream processing demonstration https://m.youtube.com/watch?v=3xw-hudsriq