cgo2007 p6 1 fulton

Performance Overheads In Real-Time Java Programs: 

Performance Overheads In Real-Time Java Programs Mark Stoodley and Mike Fulton Compilation Technology IBM Toronto Lab

Outline: 

Outline What is Real Time? Java and Real-Time Systems IBM WebSphere Real Time Overheads Some Preliminary Results Summary

What is a “Real Time” Application: 

What is a 'Real Time' Application Characterized by activities that have deadlines Often involve interactions with physical world Several facets to real-time requirements Fault tolerance: what happens when deadline missed? Level of determinism: allowable variance? Response time: how long do we have? Maxim: real-time is not just real-fast Not just embedded systems Transaction processing, complex middleware systems

Traditional Java and Real Time Systems: 

Traditional Java and Real Time Systems Real-Time systems need deterministic behaviour Predictable performance enables robust design Determinism not one of Java’s strengths Class loading, garbage collection, JIT compilation Traditional performance focus on average case Worst case performance matters more for real-time apps Must balance determinism and raw performance Customers say 'real-slow is not real-good'

The Real Time Specification for Java (RTSJ): 

The Real Time Specification for Java (RTSJ) JSR #1 Augments Java with tools to engineer RT systems Threading, scheduling, memory management, event handling, asynchrony, time, physical memory access Large and complex specification 470 pages! (JVM spec is 472 pages) No syntax changes to the language Substantial new class library support JVM implementation and OS implications

Example: Realtime and NoHeapRealtime Threads: 

Example: Realtime and NoHeapRealtime Threads RTSJ introduces new RealtimeThread class Extends java/lang/Thread Can specify scheduling policies, release parameters Also NoHeapRealtimeThread Extends RealtimeThread Created for tasks that cannot tolerate GC interruptions NHRTs not allowed to observe heap references New programmer-managed memory areas introduced Immortal, scopes

IBM WebSphere Real Time: 

IBM WebSphere Real Time Released end of August 2006 Fully compliant RTSJ implementation Built on IBM’s J9 virtual machine technology Engineered to meet customer requirements over and above what’s required by the RTSJ Significant new features: Real-time Linux kernel patches (open source model) Metronome deterministic GC Ahead-Of-Time (AOT) native code compilation

Overheads in Real-Time Native Code: 

Overheads in Real-Time Native Code Overheads for RTSJ NoHeapRealtimeThread memory checks Scope memory checks Asynchronous Transfer of Control support Overheads for Metronome GC GC is incremental so need write barriers Arraylets object model If defragmentation supported, need read barriers Not strictly 'overheads', but: Many optimizations also disabled to promote determinism Ahead-Of-Time compiled code typically slower than JITed code

NoHeapRealtimeThread (NHRT) Memory Checks: 

NoHeapRealtimeThread (NHRT) Memory Checks NHRTs cannot load heap references Exception must be thrown if heap reference found NHRT checks inserted all over the place, ahead of Parameter loads Instance and static field loads Call returns Reference array element loads Exception object load (New object allocations)

Generated Code for NHRTCheck operation: 

Generated Code for NHRTCheck operation NHRTCheck: test [ebp+#flags], #bitmask ; thread is NHRT? jz CheckDone cmp eax, andlt;heap baseandgt; jb CheckDone cmp eax, andlt;heap topandgt; ja CheckDone push ebp ; found heap ref, need to throw push eax ; MemoryAccessError exception call jitThrowMemoryAccessError CheckDone:

Why not put reference check into a snippet?: 

Why not put reference check into a snippet? Motivation: most threads are not NHRTs We discourage this thread type unless truly needed NHRTChecks are plentiful number of branches overloads processor’s BHT Processor resorts to default forward branch prediction: fall-through Only gets it right for NHRTs  Natural candidate for snippet generation

NHRTCheck operation with heap ref snippet: 

NHRTCheck operation with heap ref snippet NHRTCheck: test [ebp+#flags], #bitmask ; thread is NHRT? jnz Snippet CheckDone: … Snippet: cmp eax, andlt;heap baseandgt; jb CheckDone cmp eax, andlt;heap topandgt; ja CheckDone push ebp ; found heap ref, need to throw push eax ; MemoryAccessError exception call jitThrowMemoryAccessError

Performance Results: 

Performance Results Lower is better Higher is better

Code Size Results: 

Code Size Results Lower is better

Summary: 

Summary Real-time applications need determinism Java not traditionally suitable for RT systems RTSJ plus new technologies like Metronome GC and Ahead-Of-Time compilation making it possible Deterministic performance has overheads Many sources (RTSJ, Metronome, disabled opts) NHRT checks should be implemented in snippets Recovers some perf overhead without growing code size astronomically

Got Questions?: 

Got Questions? Mark Stoodley IBM Toronto Lab mstoodle@ca.ibm.com

Backup slides: 

Backup slides

Grand Challenge: Transparent Real-time Java: 

Grand Challenge: Transparent Real-time Java C++ Application C++ Runtime System Manual, Unsafe Predictable

WRT Architecture: 

WRT Architecture Java jar J9 VM Real-time GC Metronome Java SE 5.0 Libraries Real-time feature ('-Xrealtime') Standard J9 component X86-32 Opteron (4-way SMP) Real-time Linux (Based on RedHat 4.2) Bind Bound jar Compile Real-time JIT Real-time library

The Easy Stuff: 

The Easy Stuff Disable speculative optimizations Lower priority of JIT compiler Below priority of any real-time activity But higher than any non-real-time activity Sampling thread still has very high priority Does very little work so impact is not high Suitable for 'softer' real-time environments with 'looser' timing requirements

When JIT effects cannot be tolerated: 

When JIT effects cannot be tolerated Ahead-Of-Time compilation technology Generate native code statically Throw away platform neutrality No compilation or sampling thread active at runtime Java conformance has a performance cost All references unresolved Optimizer largely hamstrung (but not always)

Real-Time Linux: 

Real-Time Linux Customized kernel, fully open-source Fully preemptible kernel Threaded interrupt handlers for reduced latency SMP real-time scheduling High resolution timers Priority inheritance support to avoid inversion Robust and fast user-space mutex support