IBM Java Technologies

The IBM Java 5.0 SDK

Introduction and Overview

Chris Bailey
IBM Java Technologies, Hursley Park
[email protected]

WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

The Java 5.0 Release

 A Significant Release for both IBM and Sun:

– Sun specification changes:
• Large number of new Java language features
• New Java language APIs
• Introduction of JVM Tools Interface to the runtime
– IBM “Value Add” changes:
• Large number of Java Runtime Changes
• Number of RAS and Debugging Changes
• New functionality

2 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

The Java 5.0 Language

 Introduces a number of new language features:

– Generics
– Autoboxing
– Enumerated Types
– Meta Data
– Monitoring and Management
 Provided by improvements to:
– Java Class Libraries (JCL)
– Java source compiler (javac)

3 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

The IBM 5.0 Java Runtime

 Introduces a number of new features:

– New Virtual Machine Implementation
– New GC Framework
– New Just In Time (JIT) Compiler Framework
– Shared Classes on all Platforms
– New RAS Functionalities
– New RAS Tools

4 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Java Language: Generics

 Similar to C++ Templates

 Allows creation of classes in terms of one or more
generic, or unspecified, classes
 The collection classes exploit this feature
– Classes that use the collections are checked more
carefully
– Should lead to a reduction in collection based
programming mistakes

5 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Java Language: Autoboxing

 Primitive types must be “boxed” in a wrapper

class before being put into a collection
– eg. An int must be wrapped as an Integer object
 Subsequent “unboxing” must occur to use the
primitive once removed from the collection
– eg. An Integer object must be converted to an int
 Autoboxing automates this process

6 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Java Language: Enumerated Types

 Removes need to declare a sequence of named

constants as static final fields
 Looks similar to C/C++ enum:
– eg. enum Month { JANUARY, FEBRUARY, ..}
 However, contains more features:
– enum is a declaration for a fully-fledged class
• allows addition of methods and fields
• allows implementation of interfaces, ie. Comparable

7 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Java Language: MetaData

 Allows the creation of structured annotations that:

– can be attached to classes, interfaces, methods and
fields
– can be retrieved by the Java reflection API
 Allows tools that support program analysis and
application building to be more easily constructed

8 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Java Language: Monitoring and Management

 Allows Java applications or remote agents to

monitor:
– Virtual Machine “health”
– System level activities
– System level events
 Autonomic and self-adapting systems can be
developed by exploiting these features

9 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

IBM Java Runtime: Virtual Machine

 New Implementation
 History in MicroEdition (J2ME)
 Used for debugging in 1.3.1 and 1.4.2 packages
– Linux and Windows
– Invoked when using Hot Code Replace
 Shipped for some 1.4.2 64 bit platforms:
– Linux AMD64, Win AMD64 and z/OS 64

10 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Virtual Machine: History

 Began in 1996 at OTI

 Origins in SmallTalk, Visual Age for Java and
J2ME
 Strategic decision in 2002 to consolidate ME and
SE technologies
 3 years ensuring scalability and functionality
improvements made for J2SE

11 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Virtual Machine: Architecture

 Common Code base for all products (J2ME and

J2SE)
– Based on framework engines and pluggable
configurations
– Maximum code sharing and convergence on a single
code base
 Majority of Code common on all platforms
– From watches to mainframes
– Use of “builder code” for some platform specifics

12 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Virtual Machine: Value Add

 Hot Code Replacement

– In-built ability to use a debugger to change a method
and re-run that method
– Built in to original infrastructure
 High Speed Debug
– Ability to run JIT compiled code when debugging
 Data Change Breakpoints
– Ability to stop the VM when specified data value
changes

13 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

IBM Java Runtime: Garbage Collection

 New Framework Implementation

 Uses a “Type Accurate” Collector
– Doesn’t suffer from “retained garbage”
– Doesn’t suffer from pinned/dosed objects
 Introduces a parallel compactor
 Introduces a Generational GC Policy

14 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Garbage Collector: Framework Architecture

 Memory management framework

– pluggable allocation policies
– pluggable collection policies
 Four collection policies for J2SE:
– -Xgcpolicy:optthruput (default)
– -Xgcpoligy:optavgpause
– -Xgcpolicy:gencon
– -Xgcpolicy:subpools

15 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Garbage Collector: -Xgcpolicy:gencon

 Generational and Concurrect collection

– Hypothesis: objects die young
– Two generation collector:
• New generation: semi-space copy collector
• Tenured Generation: mark sweep collector
– Concurrent mark used for tenured heap

16 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Garbage Collector: -Xgcpolicy:subpools

 -Xgcpolicy:subpools
– Large SMP systems can suffer from freelist contention in
intensively multi-threaded applications
– Solution: multiple freelists
– Each freelist tracks free space of a particular size
– Available on all PPC and S390 platforms

17 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Garbage Collector: Diagnosing Problems

 Logging and Tracing functionalities:

– -verbose:gc
• output in XML
• logging to file using –Xverbosegclog:<filename>,X,Y
– -Xtgc
• garbage collection trace
• verbose and detailed output
– Flight recorder
• reduced set of GC data always logged to internal buffer
• can be extracted in failure scenarios

18 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

IBM Java Runtime: Just In Time Compiler

 New Implementation Architecture

 Consolidation with J2ME JIT
 Largely shared code on all platforms
– Optimizations done using IDL (intermediate language)
– Only final code generation is platform specific
 Still carries out compilation under –Xdebug
– Termed “full speed debug”

19 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

JIT Compiler: Differences

 1.4.2 JIT  5.0 JIT

– Compiles on the Java threads – Uses a separate thread for compiling*
– Methods are queued for compile*
– Methods are compiled on demand
– Maintains a separate stack for Java
– Uses the native stack for Java
– Can recompile methods at 5 opt
– Uses only one optimization strategy* levels*
– Asynchronously suspended for GC – Cooperatively suspended for GC
– Uses on-stack-replacement (MMI2JIT)* – Does not support on-stack-
replacement
– Method trace files are cryptic
– Method trace files are encrypted
– JIT is disabled with -Xdebug
– JIT compiles at no-opt with -Xdebug
– Written in C with some Assembler – Written in C++ with some Assembler

* Behavior can be modified by using JIT options.

20 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

JIT Compiler: Options

 Consolitdated in –Xjit: option set
– Use “–Xjit:<option1>,<option2>…” on the Java command line
– Use “IBM_JAVA_OPTIONS=-Xjit:<option1>,<option2>…” environment variable
 Option Syntax
– Applying options to a specific method set:
•-Xjit:{class.method(signature)}(<option1>,<option2>…)
– All options using {} allow regular expressions.
– {java/lang/*} - All methods within java/lang
– {java/util/[VH]*} - All java/util methods with class name beginning with V or H
– {com/[a-m]*} - All com packages that start with a through m
 Available Options
– See the Testarossa PD Guide for a complete list of options.
– http://l3.hursley.ibm.com:2211/cgi-bin/BGAuth.pl?q=/cgi-bin/ViewPRB.pl|private/2.PRB&class=private&authtype=read_bluegroup

21 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

IBM Java Runtime: Shared Classes

 Available on all platforms

 Static class data caches in shared memory
– shared between all IBM Java VMs (no master/slave)
– all application and bootstrap classes shared
– cache persisted beyond lifetime of any JVM, but lost on
shutdown/reboot
 Provides saving to memory footprint and startup
time

22 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Shared Classes

 Significant virtual memory savings when >1 JVM shares a

cache.
–Eg. ~50MB of class data with WAS (no apps).
 Significantly quicker JVM startup with a populated cache.
Classes are loaded from memory.
–Eg. Up to 36% faster starting Eclipse 3.1 on Linux
 Overhead to populate a cache is minimal
–(around 0-5% depending on application/platform)
 Class sharing is aimed at products that use multiple JVMs
concurrently or start JVMs frequently.

23 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

Using Shared Classes

 Class sharing is enabled using –Xshareclasses

– Add –Xshareclasses to application command-line
– Class cache utilities are sub-options to –Xshareclasses
– Cache size is set using –Xscmx<size>[k|m|g]
 JVM connects to an existing cache, or creates a new cache
– Multiple caches can exist, but JVM can only connect to one cache
 JVM looks for classes first in class cache…
– If class found in cache, class is loaded from cache, otherwise class is loaded from disk
and added to the cache
– No restriction on JVM classpaths, system properties or VM settings
 Multiple JVMs can concurrently populate the cache or read from it
 Custom ClassLoaders can not automatically share
– Java API and Javadoc provided for integrating custom ClassLoaders

24 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

IBM Java Runtime: RAS Functionalities and Tools

 Reworked Trace Engine:

– Activated using –Xtrace
 More flexible Dump Initiation
– Initiated using –Xdump
– More trigger events
 DTFJ Dump Analysis API

25 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: Re-engineered Trace Engine

 Configured using –Xtrace:

 “Flight Recorder” for First Failure Data Capture
(FFDC)
• Traced to in-memory buffer
• Traces key VM trace points on per thread basis
• Separate buffer for GC data
• Buffers dumped on error scenarios or user request
 Method Trace Improvements
• Trace methods entry/exit with parameters
• Trace works with JIT on or off

26 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: Trace Engine: Flight Recorder

 In case of unhandled exception, all buffers are

dumped out to a Snap file in the current working
directory
 Name like Snap0001.20051105.132657.592.trc
 This is a binary trace file that needs formatting
 Captured in addition to all normal FFDC data
 Default options in GA:
– java –Xtrace:maximal=all{level1},exception=j9mm{gclogger}

27 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: Trace Engine: Method Trace

 Enabled using mt (method trace!) component

 Two part enablement process
 1) specify destination for output, e.g print=mt or
maximal=mt
 2) select suitable methods, e.g.
methods={HelloWorld*} or
methods={java.lang.*}
– Can select methods on package, class or method name

28 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: Trace Engine: Method Trace

 Can use the accurate timestamps in formatted

binary trace to analyse performance problems in
java apps
 Parameters available with JIT enabled or disabled
 Triggering can be used to start/stop tracing on
specific events

29 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: Re-engineered Dump Engine

 Extended range of dump triggers

• From 3 triggers to 14
• Dump events extended by the use of filters
 User defined dump labels
• Ability to include: time, date, pid, uid, jre info
 Ability to configure number of dumps generated
 Ability to execute tool on dump event

30 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: Dump Engine: Trigger Events

gpf an unexpected crash, such as a SIGSEGV or a SIGILL
user SIGQUIT signal (Ctrl-Brk on Windows, Ctrl-\ on Linux, Ctrl-V on z/OS)
abort a controlled crash, as triggered by the abort() system call
vmstart the VM has finished initialization
vmstop the VM is about to shutdown
load a new class has been loaded
unload a classloader has been unloaded
throw a Java exception has been thrown
catch a Java exception has been caught
uncaught a Java exception was not handled by the application
thrstart a new thread has started
thrstop an old thread has stopped
blocked a thread is blocked entering a monitor
fullgc garbage collection has started

31 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: Dump Engine: Dump Labels

 Can use any combination of tokens and text

 Tokens expanded at time of event
– Usual date and time: %Y, %y, %m, %d, %H, %M, %S
– High precision time: %tick (msec), %seq (dump counter)
– Process info: %pid, %uid (plus %job on z/OS)
– JRE info: %home, %last (last snapped dump label)
 VM will try to create intermediate directories, for example:

/mnt/archive/dumps/%Y%m%d/%pid/javacore.%tick.txt

32 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: Dump Engine: Dump Tools

 Spawns external command

– -Xdump:tool:exec=<command string>
 Command string can contain tokens
– “%home\bin\jextract core.%Y%m%d.%H%M%S.%pid.dmp”
 Default tool attaches platform debugger to VM
– Windows: windbg
– Linux: gdb
– AIX, z/OS: dbx

33 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: DTFJ API

 Dump Toolkit and Framework for Java

 A Java API – not a Tool
 Support for post-mortem diagnosis
 Interface independent of VM implementation (J9, Sovereign)

j9core.dmp DTFJ View

DTFJ ReportController

j9core.xml

34 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: DTFJ: The API

 Top level object is Image

 Images contain Address Spaces
– Address Spaces contain Processes
• Processes (may) contain JVMs
– JVMs contain:
> Heaps
> Threads
> Locks
> …

35 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: DTFJ: Getting Started

 Get an Image Object

– There is a ImageFactory class to do this
• Different Image implementations
 Everything subsequent is based on Iterators
– Image.getAddressSpaces()
– AddressSpace.getProcesses()
– etc…

36 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: DTFJ: Uses…

 Extract information about system

–Physical memory, CPU type & count, system type
 Extract process information
–Threads, libraries & symbols, command line, environment
–Native stack traces and register contents
 Extract Java VM information
–Class loaders, threads, monitors, heaps, objects, trace buffers
–Java thread state: stack trace, priority, native thread, j.l.Thread
–Monitor state: owner, waiters
–Class info: class loader, name, inheritance, fields, methods, etc
–Object info: class, size, hashcode, id, field values

37 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: DTFJ: DTFJView

 A command-line prompt-driven tool designed to

analyze core dump files using the DTFJ API
–Look at the contents of objects
–Read arbitrary sections of memory
–Get a lot of other information from a core dump file
–Analyze problems
• e.g. detect and display deadlock information
 An extensible analysis API
 Driven by a GDB like command set

38 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

RAS: DTFJ: DTFJView: Command Set

 hexdump
 info [cls|thread|ls]
 x/[j|x|gd|wd|k]
 help
 Heap dumps, trace, synonyms, execute
 Deadlock (invokes deadlock analysis tool)

39 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation

 IBM Java Technologies

More Information…..

 Java SDK Dowloads and Information:

– http://www-128.ibm.com/developerworks/java/jdk/

 Java 5.0 Must Gather documentation:

– http://www-1.ibm.com/support/docview.wss?rs=180&uid=swg21222658

 Java 5.0 Diagnostics Guide:

– http://www-128.ibm.com/developerworks/java/jdk/diagnosis/
 IBM Java Forum:
– http://www-128.ibm.com/developerworks/forums/dw_forum.jsp?forum=367&cat=10

40 WebSphere User Group 28th Feb, 2006 © 2005 IBM Corporation