Apache Samoa ML


Presentation Description

This presentation gives an overview of the Apache Samoa ML project. It explains Apache Samoa ML in terms of it's architecture, the way that it abstracts implementation via its API and the stream processing systems that it supports. Links for further information and connecting http://www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ https://nz.linkedin.com/pub/mike-frampton/20/630/385 https://open-source-systems.blogspot.com/


Presentation Transcript

slide 1:

What Is Apache Samoa ● An Apache incubator project ● A machine learning framework ● A distributed scaleable system ● Deploys to existing Apache systems – Storm S4 Samza AVRO – Deploy a Samoa algorithm these systems – Samoa abstracts implementation via API ● Designed for stream processing ● Offers a range of ML algorithms

slide 2:

Samoa Terms Samoa terms that might be of use PE PI EPI Spout Bolt ML Processing element Processing item Entrance processing item A storm term for a data source A storm term for a data join element Machine learning

slide 3:

Samoa Algorithms ● Samoa supported algorithms – Prequential Evaluation Task – Vertical Hoeffding Tree Classifier – Adaptive Model Rules Regressor – Bagging and Boosting – Distributed Stream Clustering – Distributed Stream Frequent Itemset Mining – SAMOA for MOA users

slide 4:

Samoa Architecture

slide 5:

Samoa Architecture

slide 6:

Samoa Architecture ● The aim of Samoa is to provide implementation abstraction ● For stream processing algorithms ● Written using its API ● Against the stream processing systems that it supports ● So for instance write an algorithm once and ● Deploy to S4 and Storm ● The deployment process creates a platform jar ● That you can deploy to the specific platform

slide 7:

Samoa Topology

slide 8:

Samoa Topology ● Samoa provides a simple topology for stream processing ● This includes the elements – Processor – Content Event – Stream – Task – Topology Builder – Learner – Processing Item

slide 9:

Samoa Processor ● Processor is the basic logical processing unit ● All logic is written in the processor ● In Samoa a Processor is an interface ● Users can implement this interface – To build their own custom class ● A processor in a Samoa topology can be – A processor in the topology – An entrance processor which sources the stream

slide 10:

Samoa Content Event ● A message or an event is called Content Event in Samoa ● It is an event which contains content which ● Needs to be processed by the processors ● ContentEvent has been implemented as an interface in Samoa ● Users need to implement ContentEvent interface ● To create their custom message classes

slide 11:

Samoa Stream ● A stream is a physical unit of SAMOA topology ● Which connects different Processors with each other ● Stream is also created by a TopologyBuilder – Just like a Processor ● A stream can have a single source but many destinations ● A Processor which is the source of a stream owns the stream

slide 12:

Samoa Task ● Task is similar to a job in Hadoop ● Task is an execution entity ● A topology must be defined inside a task ● Samoa can only execute classes ● That implement Task interface

slide 13:

Samoa Topology Builder ● TopologyBuilder is a builder class ● Which builds physical units of the topology ● And assemble them together ● Each topology has a name ● An example topology might have – An EntrancePI – Some PIs – Some streams

slide 14:

Samoa Learner ● Learners are sub-topologies ● Use init function to – Add streams – Add processors – Specify connections to the topology ● Use getInputProcessor function to – Add processor that will manage the input stream ● Use getResultStream function to – Specify what is going to be the output stream

slide 15:

Samoa Processing Item ● Processing Item is a hidden physical unit of the topology ● Is just a wrapper of Processor ● It is used internally ● Is not accessible from the API ● Connects the Processor to the other processors in the topology – Simple Processing Item PI – Entrance Processing Item EntrancePI

slide 16:

Available Books ● See “Big Data Made Easy” – Apress Jan 2015 ● See “Mastering Apache Spark” – Packt Oct 2015 ● See “Complete Guide to Open Source Big Data Stack – “Apress Jan 2018” ● Find the author on Amazon – www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ ● Connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020

slide 17:

Connect ● Feel free to connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020 ● See my open source blog at – open-source-systems.blogspot.com/ ● I am always interested in – New technology – Opportunities – Technology based issues – Big data integration

authorStream Live Help