org.openide.modules.doc-files.classpath.html Maven / Gradle / Ivy
Modules, JARs, Class Loaders, and the "Class Path"
Contents
- Introduction
- The Class Loader Hierarchy
- The System Class Loader, Thread Context Class Loading, and
nbres:
- "Parallel" Libraries
- Common Problems and Solutions
Introduction
NetBeans, as a large Java application with a sophisticated module system and a
strong framework for maintaining inter-module compatibility, has a specialized
infrastructure for physically loading the classes that produce the
application. It uses class loaders to manage the interactions between
modules.
While developers who are accustomed to working with application servers or
other large componentized applications might already be familiar with the
purposes and basic mechanisms of class loader partitioning, many Java
developers only have experience with monolithic applications loaded entirely
from the system class path (the -classpath parameter to
the Java VM launcher). All developers seeking to write a NetBeans module
should have some practical understanding of how NetBeans manages class
loading.
This document attempts to explain the basic system by which classes are loaded
into NetBeans; and how you as a module author can take advantage of the power
this system offers, without becoming a victim to some common misperceptions
and mistakes.
Everything written here is nonnormative, which is to say it is not a
formal part of the Modules API specification. For precise guarantees as to
what you can and cannot do, as well as details on several points of manifest
syntax, please read the
Modules API.
The Class Loader Hierarchy
The basic thing you need to understand about how modules control class loading
is this:
If module B has a declared dependency on module A, then classes in B can
refer to classes in A (but A cannot refer to B). If
B does not have a declared dependency on A, it cannot refer to A.
Furthermore, dependencies are not considered transitive for purposes of
classloading: if C has a declared dependency on B, it can refer to classes in
B, but not to A (unless it also declares an explicit dependency on A).
Remember: refer to here means refer to statically: compile
and link against. Source code for B should be passed the JAR for A in its
classpath when compiling. Module A can certainly use objects created by module
B, if assigned to a suitably generic interface; it just cannot link against
these specific classes in Java source code.
The same system applies to classpath-oriented calls such as
Class.getResource(String path).
This constraint is enforced using the parent-child relationship of class
loaders. Each module has its own dedicated class loader. A NetBeans
module classloader delegates only to the classloaders of modules which the
module author asked to depend on.
This kind of relationship - a hierarchy of classloaders - is common in
componentized Java applications. NetBeans takes the concept a little bit
further: since a module can (and very often will) depend on several other
modules, the classloader hierarchy is a directed acyclic graph, rather than a
tree. This could be called multiple inheritance of classloaders.
There are a number of reasons for using such a system, rather than dumping
everything in one big classpath. The main reason, though, is safety. If
NetBeans let you use any class from any other module, you might begin relying
on another module without even realizing it. This could cause sudden errors if
that other module were missing from a user's installation. Declaring a module
dependency prevents such a situation from arising.
Here is a trivial summary of the situation:
Manifest-Version: 1.0
OpenIDE-Module: org.netbeans.modules.a
public abstract class A {/* ... */}
// ...
public static void useSomeA(A a) {/* ... */}
Manifest-Version: 1.0
OpenIDE-Module: org.netbeans.modules.b
OpenIDE-Module-Module-Dependencies: org.netbeans.modules.a
public class B extends A {/* ... */}
// ...
useSomeA(new B());
This basic rule covers the bulk of the situations you will encounter. There
are some further reasons why you might not be able to access certain classes,
but you can probably stop reading now unless you actually run into problems.
Extensions using Class-Path
Commonly a NetBeans module will serve as a wrapper for an independent library,
or will use one or more separate libraries as part of its implementation or
API. Forcing all classes and resources used by a module to be packed together
into the module JAR is generally undesirable for reasons of maintenance and
clarity. Sometimes it may even violate license terms of a library to modify it
by repackaging it into a module JAR.
NetBeans handles this situation by letting you use the Class-Path
manifest attribute defined by the
Java Extension Mechanism.
The value of the attribute consists of one or more relative paths to library
JARs, in the same directory as the module or (normally) beneath it.
Conventionally, such extensions are placed in a directory named
ext/ beneath the directory where the module is. It is a good idea
to name extension JARs precisely (i.e., include the version number).
For example, a module $cluster/modules/module.jar would
refer to the extensions
$cluster/modules/ext/library-a-1.0.jar and
$cluster/modules/ext/library-b-1.1.jar this way:
Class-Path: ext/library-a-1.0.jar ext/library-b-1.1.jar
When you do this, the libraries are loaded from the same classloader as the
main module JAR, just as if they had been unpacked and physically added to it.
Classes from any of these JARs can refer to classes in the others. As a matter
of style, however, you should generally refer to the libraries from the module
but not refer to the module from the libraries. Modules dependent on a module
with extensions can refer to its extensions too.
It is common for a module to just be a wrapper which has no classes
itself, but merely refers to a library using Class-Path. Creating
such a wrapper module blesses the library with the flexible deployment and
maintenance qualities of a module. If there is no legal reason not to, you can
also just insert NetBeans-specific manifest attributes in any JAR and
use it as a module, without interfering with its use outside NetBeans.
Overuse of Class-Path is a common bad habit. Use module
dependencies to express relationships between independent blocks of code;
Class-Path should be used only when a JAR is a wholly owned part
of a module.
Never refer to the same JAR using Class-Path
from two different modules. (It will get loaded twice and could cause
strange problems.) If you think you need to, generally this just means you
have not factored out your module dependencies thoroughly: split off a third
module to hold the library.
Never try to use Class-Path to refer to other
module JARs, or to JARs in the NetBeans lib/ directory, or
generally to any JAR you did not specifically bundle alongside your module and
your module only (for example, in an NBM package for Auto Update).
Do not be alarmed by all the "never"s; these are good guidelines to protect
you from common design mistakes, but the basic behavior of
Class-Path is pretty simple:
If a module A has a Class-Path attribute listing some relative
JAR paths, the effect is exactly the same as if the contents of those JARs
were just copied into A's main JAR.
Startup Libraries
In a given NetBeans release, there are several libraries placed in the
lib/ and core/ subdirectories of the NetBeans
installation. These are referred to as startup libraries.
If you need to use classes from them, just declare regular module dependencies.
Patches and Localizations
In addition to use of Class-Path, there are a couple of other
ways in which additional JARs might be added into the classloader for a
module.
Patches are JARs which may replace classes and resources in a module
classloader. If a module has the code name base a.b.c, and there
is a subdirectory named patches/a-b-c/ beneath the directory in
which the module JAR resides, then this patch area will be scanned for JAR
files to be added to the "front" of the module classloader - anything found in
them will override corresponding classes or resources in the module and its
extensions.
This patching facility is occasionally useful for testing modifications to
installed modules, or making emergency fixes in the field, without needing to
touch the master copy of the module. Modules should not be shipped with any
patches, however.
Localization and branding may also affect the module classloader. A
module may load resources (such as property bundles, images, XML layers, and
so on) using the recommended internationalized lookup techniques -
NbBundle methods, the nbresloc: URL protocol, and so
on. Actual localizations of these resources are best placed in separate JAR
files, which must be named according to the locale and placed in a
subdirectory named locale/ beneath the directory in which the
module JAR resides. For example, a Japanese localization of a module
$cluster/modules/module.jar would be named
$cluster/modules/locale/module_ja.jar.
Analogously, a module may be branded to adjust it to use in a
specific product, using essentially the same mechanism. The module just
mentioned could have a few text changes made for a product branding named
myapp by creating a file named
$cluster/modules/locale/module_myapp.jar. Locales and
brandings are orthogonal, so a Belgian French translation of text strings
specific to MyApp Community Edition might be kept in
$cluster/modules/locale/module_myapp_ce_fr_BE.jar.
All applicable locale and branding variants of module JARs are placed in the
same classloader as the module itself. Only variant JARs which actually apply
at runtime (according to the current locale and branding) will be loaded.
Provide-Require Dependencies
In addition to regular dependencies using
OpenIDE-Module-Module-Dependencies, modules can also depend on
the existence of other modules using provide-require dependencies.
Here, one or more modules provide some arbitrary token (often the name of an
API class for which they supply implementations in Lookup); and other modules
may request that this token be available. A requesting modules can be enabled
only when at least one (perhaps more) providing module is enabled.
Provide-require dependencies have no effect on classloading. If module A
provides token T, and module B requires token T, B may not refer to classes
in A (unless it additionally declares a regular dependency on A).
Dependencies and Package Restrictions
By default when a module B declares a direct dependency on another module A,
then B can access any public (or, as appropriate, protected) class in A - i.e.
all of A is "in its classpath". However it is often the case that such broad
access is undesirable. Module A may provide a formal API in certain packages,
and use other packages for its private implementation. In this case it would
not want other modules freely using its undocumented implementation classes.
For this reason, a special manifest attribute
OpenIDE-Module-Public-Packages can be specified in A:
Manifest-Version: 1.0
OpenIDE-Module: org.netbeans.modules.a
OpenIDE-Module-Specification-Version: 1.0
OpenIDE-Module-Implementation-Version: 1.0-alpha-2
OpenIDE-Module-Public-Packages: org.netbeans.api.a.**, org.netbeans.spi.a.**
This attribute tells the NetBeans module system to limit which packages from A
are considered part of its API. If B declares a regular dependency on A
according to its public API specification version:
Manifest-Version: 1.0
OpenIDE-Module: org.netbeans.modules.b
OpenIDE-Module-Module-Dependencies: org.netbeans.modules.a > 1.0
then B can only use some packages from A: here,
org.netbeans.api.a (and any subpackages it may have) and
org.netbeans.spi.a (and subpackages). Module B will not
be permitted to use other packages from A, such as
org.netbeans.modules.a; attempts to do so will just result in a
NoClassDefFoundError when loading code from B.
This manifest goes further and prevents any packages from being
available to other modules:
Manifest-Version: 1.0
OpenIDE-Module: org.netbeans.modules.a
OpenIDE-Module-Public-Packages: -
You may still declare a dependency on such a module, in order to ensure that
it is installed (perhaps it provides some non-Java-level service you need),
but you may not import any classes from it.
There is a limited "back door" to this package restriction: if module B
declares that it knows about module A's internal implementation, and is
prepared to track arbitrary changes in A, then it can use any public classes
from A regardless of the public package declaration. This is done using an
implementation dependency:
Manifest-Version: 1.0
OpenIDE-Module: org.netbeans.modules.b
OpenIDE-Module-Module-Dependencies: org.netbeans.modules.a = 1.0-alpha-2
Here module B declares that it is written to match details of A's
implementation classes at that point in time. This version of B may not use
any earlier or later versions of A, since there is no telling what
changes to A's implementation classes there might have been. Therefore, such
implementation dependencies are usually used among "friend modules" which are
updated and published in clusters by a single developer or team.
Again, dependencies are not considered transitive for purposes of class
loading; so package visibility from module A to module C (where C depends on
B) is entirely independent of visibility from A to B - it can be computed
entirely from the manifests of A and C.
Public package declarations also apply to packages contained in
Class-Path extensions - just as if those classes were in the main
module JAR.
In summary:
If module A declares that it has a particular list of public packages, and
module B declares a direct dependency on A, B can always use classes from
A's public packages (if any). B may only use undeclared packages from A
(implementation classes) if it declares an implementation dependency on the
exact version of A.
The System Class Loader, Thread Context Class Loading, and nbres:
There is a special class loader in NetBeans referred to as the system
class loader. This loader can load classes from any enabled module, as
well as the JRE/JDK and all startup libraries (APIs and core).
Therefore it is useful as a default class
loader to be used by any code which tries to find a class by name without
specific knowledge of where it may be located.
Finding the system class loader is easy using lookup:
ClassLoader syscl = Lookup.getDefault().lookup(ClassLoader.class);
This class loader is also the
context class loader
for every thread in the NetBeans VM, unless that thread (or a parent)
explicitly set some other context class loader. Since many libraries which are
independent of NetBeans (including in the JRE) are written to assume that all
relevant classes can be loaded by name from the current thread's context loader,
it is very useful for this loader to be the system class loader - you can specify
any class in your module by name.
ClassLoader l = Thread.currentThread().getContextClassLoader();
Class c = l.loadClass("some.module.Class");
Any module class can be accessed from this loader, regardless of any
OpenIDE-Module-Public-Packages declarations.
The system loader always represents the contents of the enabled modules in
NetBeans. If a module is newly enabled at runtime, its classes are effectively
added to the namespace of the same system ClassLoader
instance. (Note that this means that a call to Class.forName
which fails before the module is enabled may succeed afterwards.) However, if
a module is disabled at runtime, the system class loader is
reset to a new loader which does not have access to the old module.
This is necessary because it is impossible to remove classes from a loader
once they have been loaded. After a module is disabled and the loader reset, a
saved Lookup.Result query on ClassLoader will fire a
lookup result change, and all threads will be updated to get the new context
class loader too.
It is often useful to be able to refer to resources other than
classes in the system class loader. This is easy to do because NetBeans
defines a special URL protocol handler for just this purpose. URLs with the
nbres protocol refer to resources in the system loader, and thus
can refer to resources present in any module JAR. This is very useful when
some declarative syntax requires a URL that should look in a module; for
example:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE filesystem PUBLIC "-//NetBeans//DTD Filesystem 1.1//EN"
"http://www.netbeans.org/dtds/filesystem-1_1.dtd">
<!-- Register one DTD in the system entity catalog. -->
<filesystem>
<folder name="xml">
<folder name="entities">
<folder name="NetBeans">
<file name="DTD_Foo_1_0"
url="nbres:/org/netbeans/modules/foo/resources/foo-1.0.dtd">
<attr name="hint.originalPublicID"
stringvalue="-//NetBeans//DTD Foo 1.0//EN"/>
</file>
</folder>
</folder>
</folder>
</filesystem>
There is also a related protocol nbresloc which loads from the
system class loader but additionally performs automatic localization and
branding of the resource you specify. Various
suffixes
are inserted between the base name and the extension of the resource
(beginning with the last dot in the path, if it is in the last path
component), according to the current locale and branding. For example:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE filesystem PUBLIC "-//NetBeans//DTD Filesystem 1.1//EN"
"http://www.netbeans.org/dtds/filesystem-1_1.dtd">
<!-- Add a URL to the Help menu with a localized name and icon. -->
<filesystem>
<folder name="Menu">
<folder name="Help">
<file name="netbeans-web-link.url" url="netbeans-web-link.url">
<attr name="SystemFileSystem.localizingBundle"
stringvalue="org.netbeans.modules.url.Bundle"/>
<attr name="SystemFileSystem.icon"
urlvalue="nbresloc:/org/netbeans/modules/url/webLink.gif"/>
</file>
</folder>
</folder>
</filesystem>
Here NetBeans will look for the most specific applicable icon; it might for
example find org/netbeans/modules/url/webLink_ja.gif in
modules/locale/utilities_ja.jar, if Japanese localizers decided
the default icon was not intuitive for Japanese users and made a replacement.
Whenever you are asked for a URL to a displayable resource, consider using
nbresloc in place of nbres.
"Parallel" Libraries
It is possible for two modules to include classes with the same (fully-qualified) names, so long as the modules have distinct code name bases.
This can be useful in case you want to ship several versions of a third-party library.
For example, you could have modules org.apache.log4j.v1 and org.apache.log4j.v2
both including org.apache.log4j.** classes and exposing these packages as public.
Some modules can depend on and use version 1 while other modules depend on and use version 2.
Potential problems you might run into:
-
The context class loader will refuse to load any of the duplicated classes.
(It could not know which you meant to load.)
You would need to pass a specific ClassLoader to any code which needed it.
-
nbres-protocol URLs will similarly not work for duplicated resources.
-
Some third-party libraries, including Xerces, perversely use the thread context class loader
to load some classes which are in their own JARs, even though they could just use the class
loader which loaded the calling code. Such calls will fail. You can work around this by
temporarily setting the thread CCL to that of the desired library module, restoring it
in a finally block.
-
A given module cannot depend on both of these library modules at once - it would not know which to load.
Common Problems and Solutions
This section is an attempt to gather a number of the problems, questions, and
misperceptions that people using the NetBeans module system have run across.
In some cases an error is caused by a simple mistake that can be corrected
just as simply; in other cases, you may need to consider the design of your
module and how it can work smoothly in NetBeans.
The developer's FAQ is likely to have more recent answers to your questions.
I am getting warnings when trying to access resources from the default (root, unnamed) package
Loading classes from the default (root) package is disabled. You can load resources from
that package but a warning is printed. It is strongly discouraged to use the default package.
Please see the
Java Language Specification for more details.
You can suppress the warning using -J-Dorg.netbeans.ProxyClassLoader.level=1000 command
line switch.
I would like to use %CLASSPATH% / $CLASSPATH
NetBeans ignores %CLASSPATH% / $CLASSPATH environment variables
and defines its own classpath containing minimal set of classes required to start
the application using -classpath parameter passed to JVM.
All other classes are loaded by classloaders created during runtime and briefly described
in this document.
If it is really neceseary to add more classes or resources to application classloader
--cp:a or --cp:p options of launcher can be used.
Using lib/*.jar or -cp sounds much easier but someone told me not to do it
The reason is similar to why JDK documentation about setting the class path for
Windows or
Solaris
states that -classpath is prefered over environment variables.
Adding classes (resources) to lib/ext/ affects all Java applications
running using this Java installation.
Use of --cp affects all modules and for example prevents the possibility
to have more modules depending on different version of the same library.
I need to add a library JAR from outside the NetBeans installation
Q: Can my module add a library JAR to the classpath from outside the
IDE installation? For example, I have an application called Etch-a-Kvetch for
modeling customer call response systems, and I want to build a module to
integrate it into the IDE. The user may already have Etch-a-Kvetch installed
on their disk, and I want to reuse the eak.jar main library JAR
in my module. It is not present in the IDE's installation directory. Can I add
it to the IDE's classpath so my module can use it?
A: Not easily. You have a few options:
-
Add an entry to ide.cfg. For example:
-cp:a c:\eak\lib\eak.jar
This startup file provides the ability to add classpath entries to the IDE's
Java invocation.
Pros: Adds the library as desired. Cons: Very fragile.
Assumes that the ide.cfg is actually read, which is not
guaranteed for all platforms. Easy to clobber existing user customizations.
Dependent on the exact structure of the IDE's bin/ directory,
which has changed quite a bit in the past and could change again. Places
library in startup classpath, whereas it is preferred to have it in the
module classloader only. Must be done before the IDE starts, requiring some
kind of manual installation step and making updating your module very
difficult.
-
Duplicate eak.jar in modules/ext/. That
is, ship a copy of the required JAR file inside the IDE installation, and
continue to refer to dynamic resources in the desired external location.
Pros: Simple to implement and should be reliable. Version of the
library shipped can be controlled to match that which the module was
compiled against, avoiding potential version skew. Cons:
Impractical when the library is physically very large, or there are
licensing issues involved in redistributing it. Bugfix upgrades to the
master library may not be reflected in the IDE's copy.
-
Partition your module and use a new classloader. In other words:
logically divide your module into two halves. The first half will form a
compilation unit depending on the Open APIs and will contain all of the
classes referred to directly in the module manifest (the installer, data
loaders, or whatever you have). It should contain one or more interfaces or
similar constructs which describe what it expects the second half to
implement. The second half will form a separate compilation unit, depending
on the first half, possibly the Open APIs, and also eak.jar. It
should contain implementations of the interfaces specified in the first
half. The first half, at install/restore time (or lazily when any
functionality is needed) should create a new URLClassLoader
whose parent should be the first half's classloader, and including as URLs
the locations of both the second half as a JAR and the library JAR; using
this classloader, look up the implementation classes by name; create new
instances of them; cast them to the interface types; and begin using them.
The second half should probably be placed in a separate JAR file; if not, it
will be necessary to subclass the dynamic classloader to not delegate class
loads for the implementation packages to its parent. Either way, it is
strongly recommended that the build process enforce that the first half
compile without reference to the second.
Pros: Compliant with the Open APIs and reliable. The library JAR
may be located anywhere at runtime and could even be moved or replaced at
runtime. Cons: Potentially complex to implement. Some use of
reflection required, though it should be safe. More complex build and test
procedure for the module. The partition between the two halves must be
carefully chosen, especially for large modules, to minimize the complexity
of the interfaces and push as much implementation as possible to one side or
the other.
Oyetunde Fadele has kindly posted a detailed explanation of how to use this
technique in practice: see his message long ago on [email protected].
@FOOTER@