org.apache.juneau.urlencoding.package.html Maven / Gradle / Ivy
URL encoding serialization and parsing support
Table of Contents
-
-
-
1 - URL encoding support overview
Juneau supports converting arbitrary POJOs to and from URL-encoded strings using ultra-efficient serializers
and parsers.
The serializer converts POJOs directly to URL-encoded strings without the need for intermediate DOM objects
using a highly-efficient state machine.
Likewise, the parser creates POJOs directly from URL-encoded strings without the need for intermediate DOM
objects.
Juneau uses UON (URL-Encoded Object Notation) for representing POJOs.
The UON specification can be found here.
Juneau can serialize and parse instances of any of the following POJO types:
-
Java primitives and primitive objects (e.g.
String
, Integer
, Boolean
,
Float
).
-
Java Collections Framework objects (e.g.
HashSet
, TreeMap
) containing anything
on this list.
-
Multi-dimensional arrays of any type on this list.
-
Java Beans with properties of any type on this list.
-
Classes with standard transformations to and from
Strings
(e.g. classes containing
toString()
, fromString()
, valueOf()
, constructor(String)
).
-
Non-serializable classes and properties with associated
PojoSwaps
that convert them to
serializable forms.
Refer to POJO Categories
for a complete definition of supported POJOs.
Prerequisites
The Juneau URL-encoding serialization and parsing support does not require any external prerequisites.
It only requires Java 1.6 or above.
1.1 - URL-encoding support overview - example
The example shown here is from the Address Book resource located in the
org.apache.juneau.sample.war
application
The POJO model consists of a List
of Person
beans, with each Person
containing zero or more Address
beans.
When you point a browser at /sample/addressBook/people/1
, the POJO is rendered as HTML:
By appending ?Accept=application/x-www-form-urlencoded&plainText=true
to the URL, you can
view the data as a URL-encoded string:
0 =(
uri =http://localhost:10000/addressBook/people/1 ,
addressBookUri =http://localhost:10000/addressBook/ ,
id =1 ,
name ='Barack+Obama' ,
birthDate ='Aug+4,+1961' ,
addresses =@(
(
uri =http://localhost:10000/addressBook/addresses/1 ,
personUri =http://localhost:10000/addressBook/people/1 ,
id =1 ,
street ='1600+Pennsylvania+Ave' ,
city =Washington ,
state =DC ,
zip =20500 ,
isCurrent =true
),
(
uri =http://localhost:10000/addressBook/addresses/2 ,
personUri =http://localhost:10000/addressBook/people/1 ,
id =2 ,
street ='5046+S+Greenwood+Ave' ,
city =Chicago ,
state =IL ,
zip =60615 ,
isCurrent =false
)
),
age =56
)
&1 =(
uri =http://localhost:10000/addressBook/people/2 ,
addressBookUri =http://localhost:10000/addressBook/ ,
id =2 ,
name ='George+Walker+Bush' ,
birthDate ='Jul+6,+1946' ,
addresses =@(
(
uri =http://localhost:10000/addressBook/addresses/3 ,
personUri =http://localhost:10000/addressBook/people/2 ,
id =3 ,
street ='43+Prairie+Chapel+Rd' ,
city =Crawford ,
state =TX ,
zip =76638 ,
isCurrent =true
),
(
uri =http://localhost:10000/addressBook/addresses/4 ,
personUri =http://localhost:10000/addressBook/people/2 ,
id =4 ,
street ='1600+Pennsylvania+Ave' ,
city =Washington ,
state =DC ,
zip =20500 ,
isCurrent =false
)
),
age =71
)
Juneau supports two kinds of serialization:
-
Construction of full URL query parameter strings (e.g.
&key=value
pairs) from beans
and maps.
-
Construction of URL query parameter value strings (e.g. just the
value
portion of
&key=value
pairs) from any POJO.
Top-level beans and maps can serialized as key/value pairs as shown below:
Example: A bean with 2 string properties, 'foo' and 'baz', serialized to a query string
http://localhost/sample?foo =bar &baz =bing
Lower-level beans and maps are also serialized as key/value pairs, but are surrounded with a
"(...)" construct to denote an object mapping, and uses a comma as the parameter delimiter
instead of "&" .
Example: A bean serialized as a query parameter value.
http://localhost/sample?a1 =(foo =bar ,baz =bing )
General methodology:
Java type JSON equivalent UON
Maps/beans
OBJECT
a1 =(b1 =x1 ,b2 =x2 )
a1 =(b1 =(c1 =x1 ,c2 =x2 ))
Collections/arrays
ARRAY
a1 =@(x1 ,x2 )
a1 =@(@(x1 ,x2 ),@(x3 ,x4 ))
a1 =@((b1 =x1 ,b2 =x2 ),(c1 =x1 ,c2 =x2 ))
Booleans
BOOLEAN
a1 =true &a2 =false
int/float/double/...
NUMBER
a1 =123 &a2 =1.23e1
null
NULL
a1 =null
String
STRING
a1 =foobar
a1 ='true'
a1 ='null'
a1 ='123'
a1 =' string with whitespace '
a1 ='string with ~'escaped~' quotes'
Refer to the UON specification for a complete set of syntax rules.
PojoSwaps
can be used to convert non-serializable POJOs into serializable forms, such as
converting Calendar
object to ISO8601 strings, or byte []
arrays to
Base-64 encoded strings.
These transforms can be associated at various levels:
- On serializer and parser instances to handle all objects of the class type globally.
- On classes through the
@Bean
annotation.
- On bean properties through the
@BeanProperty
annotations.
Example: A serialized Calendar object using CalendarSwap.RFC2822DTZ
transform.
http://localhost/sample?a1='Sun,+03+Mar+1901+09:05:06+GMT'
For more information about transforms, refer to org.apache.juneau.transform.
2 - UrlEncodingSerializer and UonSerializer classes
{@link org.apache.juneau.urlencoding.UrlEncodingSerializer} and {@link org.apache.juneau.uon.UonSerializer}
classes are used to convert POJOs to URL-encoded strings.
The UonSerializer
class converts parameter values to UON notation.
The UrlEncodingSerializer
class converts a POJO to key/value URL-Encoded pairs using
UonSerializer
to serialize the values.
If you're trying to construct complete URL-Encoded entities, use UrlEncodingSerializer
.
If you're constructing your own key/value pairs, use UonSerializer
.
The serializers include several configurable settings.
Static reusable instances of serializers are provided with commonly-used settings:
-
{@link org.apache.juneau.urlencoding.UrlEncodingSerializer#DEFAULT}
- All default settings, strict mode.
-
{@link org.apache.juneau.urlencoding.UrlEncodingSerializer#DEFAULT_READABLE}
- Use whitespace and indentation for readability.
-
{@link org.apache.juneau.uon.UonSerializer#DEFAULT}
- All default settings, strict mode.
-
{@link org.apache.juneau.uon.UonSerializer#DEFAULT_READABLE}
- Use whitespace and indentation for readability.
-
{@link org.apache.juneau.uon.UonSerializer#DEFAULT_ENCODING}
- Same as DEFAULT, but use URL-Encoding on special characters.
The general guidelines on which serializer to use is:
-
Use encoding serializers when you're using the results to construct a URI yourself, and therefore
need invalid URI characters to be encoded.
-
Use un-encoding serializers when you're creating parameter values and passing them off to some other
utility class that will itself encode invalid URI characters.
-
Use the readable serializer for debugging purposes.
Notes about examples
The examples shown in this document will use default strict settings.
For brevity, the examples will use public fields instead of getters/setters to reduce the size of the examples.
In the real world, you'll typically want to use standard bean getters and setters.
To start off simple, we'll begin with the following simplified bean and build upon it.
public class Person {
// Bean properties
public int id ;
public String name ;
// Bean constructor (needed by parser)
public Person() {}
// Normal constructor
public Person(int id, String name) {
this .id = id;
this .name = name;
}
}
The following code shows how to convert this to a URL-encoded value:
// Use serializer with readable output, simple mode.
UonSerializer s = UonSerializer.DEFAULT ;
// Create our bean.
Person p = new Person(1, "John Smith" );
// Serialize the bean to URL-encoded parameter value.
String urlencoded = s.serialize(p);
The code above produces the following output:
(id =1 ,name ='John+Smith' )
The {@link org.apache.juneau.urlencoding.UrlEncodingSerializer} class converts
maps and beans into top-level query parameter strings.
// Use serializer with readable output, simple mode.
UrlEncodingSerializer s = UrlEncodingSerializer.DEFAULT ;
// Serialize the bean to URL-encoded query string.
String urlencoded = s.serialize(p);
The code above produces the following output:
id =1 &name ='John+Smith'
By default, the UrlEncodingSerializer
class will URL-Encode special characters, and the
UonSerializer
will NOT URL-encode special characters.
2.1 - @Bean and @BeanProperty annotations
The {@link org.apache.juneau.annotation.Bean @Bean} and {@link org.apache.juneau.annotation.BeanProperty @BeanProperty}
annotations are used to customize the behavior of beans across the entire framework.
They have various uses:
- Hiding bean properties.
- Specifying the ordering of bean properties.
- Overriding the names of bean properties.
- Associating transforms at both the class and property level (to convert non-serializable POJOs to
serializable forms).
For example, we now add a birthDate
property, and associate a transform with it to transform
it to an ISO8601 date-time string in GMT time.
We'll also add a couple of URI
properties.
By default, Calendars
are treated as beans by the framework, which is usually not how you want
them serialized.
Using transforms, we can convert them to standardized string forms.
public class Person {
// Bean properties
public int id ;
public String name ;
public URI uri ;
public URI addressBookUri ;
@BeanProperty (swap=CalendarSwap.ISO8601DTZ.class ) public Calendar birthDate ;
// Bean constructor (needed by parser)
public Person() {}
// Normal constructor
public Person(int id, String name, String uri, String addressBookUri, String birthDate)
throws Exception {
this .id = id;
this .name = name;
this .uri = new URI(uri);
this .addressBookUri = new URI(addressBookUri);
this .birthDate = new GregorianCalendar();
this .birthDate
.setTime(DateFormat.getDateInstance (DateFormat.MEDIUM )
.parse(birthDate));
}
}
Next, we alter our code to pass in the birthdate:
// Create our bean.
Person p = new Person(1, "John Smith" , "http://sample/addressBook/person/1" ,
"http://sample/addressBook" , "Aug 12, 1946" );
Now when we rerun the sample code, we'll get the following:
(id =1 ,name ='John+Smith' ,uri =http://sample/addressBook/person/1 ,
addressBookUri =http://sample/addressBook ,birthDate =1946-08-12T00:00:00Z )
Using UrlEncodingSerializer
instead would create the following:
id =1 &name ='John+Smith' &uri =http://sample/addressBook/person/1
&addressBookUri =http://sample/addressBook &birthDate =1946-08-12T00:00:00Z
Another useful feature is the {@link org.apache.juneau.annotation.Bean#propertyNamer()} annotation that
allows you to plug in your own logic for determining bean property names.
The {@link org.apache.juneau.PropertyNamerDLC} is an example of an alternate property namer.
It converts bean property names to lowercase-dashed format.
Example:
@Bean (propertyNamer=PropertyNamerDLC.class )
public class Person {
...
Results
(id =1,name ='John+Smith' ,uri =http://sample/addressBook/person/1 ,
address-book-uri =http://sample/addressBook ,birth-date =1946-08-12T00:00:00Z )
2.2 - Collections
In our example, let's add a list-of-beans property to our sample class:
public class Person {
// Bean properties
public LinkedList<Address> addresses = new LinkedList<Address>();
...
}
The Address
class has the following properties defined:
public class Address {
// Bean properties
public URI uri ;
public URI personUri ;
public int id ;
public String street , city , state ;
public int zip ;
public boolean isCurrent ;
}
Next, add some quick-and-dirty code to add an address to our person bean:
// Use serializer with readable output, simple mode.
UonSerializer s = UonSerializer.DEFAULT_READABLE ;
// Create our bean.
Person p = new Person(1, "John Smith" , "http://sample/addressBook/person/1" ,
"http://sample/addressBook" , "Aug 12, 1946" );
Address a = new Address();
a.uri = new URI("http://sample/addressBook/address/1" );
a.personUri = new URI("http://sample/addressBook/person/1" );
a.id = 1;
a.street = "100 Main Street" ;
a.city = "Anywhereville" ;
a.state = "NY" ;
a.zip = 12345;
a.isCurrent = true ;
p.addresses .add(a);
Now when we run the sample code, we get the following (in readable format):
(
id =1 ,
name ='John+Smith' ,
uri =http://sample/addressBook/person/1 ,
addressBookUri =http://sample/addressBook ,
birthDate =1946-08-12T00:00:00Z ,
addresses =@(
(
uri =http://sample/addressBook/address/1 ,
personUri =http://sample/addressBook/person/1 ,
id =1 ,
street ='100+Main+Street' ,
city =Anywhereville ,
state =NY ,
zip =12345 ,
isCurrent =true
)
)
)
If we were to use UrlEncodingSerializer
instead, we would get the following:
id =1 &
name ='John+Smith' &
uri =http://sample/addressBook/person/1 &
addressBookUri =http://sample/addressBook &
birthDate =1946-08-12T00:00:00Z &
addresses =@(
(
uri =http://sample/addressBook/address/1 ,
personUri =http://sample/addressBook/person/1 ,
id =1 ,
street ='100+Main+Street' ,
city =Anywhereville ,
state =NY ,
zip =12345 ,
isCurrent =true
)
)
Note how the top level Person
bean is serialized using the standard '&' delimiter,
whereas the lower-level Address
bean is serialized using the ',' character to prevent
the addresses
field from being incompletely parsed.
2.3 - Non-tree models and recursion detection
The URL-encoding serializer is designed to be used against POJO tree structures.
It expects that there not be loops in the POJO model (e.g. children with references to parents, etc...).
If you try to serialize models with loops, you will usually cause a StackOverflowError
to
be thrown (if {@link org.apache.juneau.serializer.Serializer#SERIALIZER_maxDepth} is not reached first).
If you still want to use the URL-encoding serializer on such models, Juneau provides the
{@link org.apache.juneau.serializer.Serializer#SERIALIZER_detectRecursions} setting.
It tells the serializer to look for instances of an object in the current branch of the tree and
skip serialization when a duplicate is encountered.
For example, let's make a POJO model out of the following classes:
public class A {
public B b;
}
public class B {
public C c;
}
public class C {
public A a;
}
Now we create a model with a loop and serialize the results.
// Clone an existing serializer and set property for detecting recursions.
UrlEncodingSerializer s = UrlEncodingSerializer.DEFAULT_READABLE .builder()
.detectRecursions(true ).build();
// Create a recursive loop.
A a = new A();
a.b = new B();
a.b .c = new C();
a.b .c .a = a;
// Serialize.
String json = s.serialize(a);
What we end up with is the following, which does not serialize the contents of the c
field:
(
b =(
c =()
)
)
Without recursion detection enabled, this would cause a stack-overflow error.
Recursion detection introduces a performance penalty of around 20%.
For this reason the setting is disabled by default.
2.4 - Configurable properties
See the following classes for all configurable properties that can be used on this serializer:
-
{@link org.apache.juneau.BeanContext}
- Bean context properties.
-
{@link org.apache.juneau.uon.UonSerializerContext}
- UON serializer context properties.
-
{@link org.apache.juneau.urlencoding.UrlEncodingSerializerContext}
- URL-Encoding serializer context properties.
2.5 - Other notes
-
Like all other Juneau serializers, the URL-encoding serializers are thread safe and maintain an internal
cache of bean classes encountered.
For performance reasons, it's recommended that serializers be reused whenever possible instead of
always creating new instances.
3 - UrlEncodingParser and UonParser classes
{@link org.apache.juneau.urlencoding.UrlEncodingParser} and {@link org.apache.juneau.uon.UonParser} classes
are used to convert URL-encoded strings back into POJOs.
The UonParser
class converts UON-encoded parameter values to POJOs.
The UrlEncodingParser
class converts entire URL-Encoded strings to POJOs using
UonSerializer
to serialize individual values.
If you're trying to parse an entire URL-Encoded string, use UrlEncodingParser
.
If you're trying to parse an individual value (such as that returned by
RestServlet.getQueryParameter(name)
), use UonParser
.
The following static reusable instances of UrlEncodingParser
are provided for convenience:
-
{@link org.apache.juneau.urlencoding.UrlEncodingParser#DEFAULT}
- Default parser for entire URL-encoded strings, decode
%xx
sequences.
-
{@link org.apache.juneau.uon.UonParser#DEFAULT}
- Default parser for URL-encoded parameter values, don't decode
%xx
sequences.
-
{@link org.apache.juneau.uon.UonParser#DEFAULT_DECODING}
- Default parser for URL-encoded parameter values, decode
%xx
sequences.
The general guidelines on which parser to use is:
-
Use the
DEFAULT
parser for parameter values that have already had %xx
sequences
decoded, such as when using HttpServletRequest.getQueryParameter(name)
.
-
Use the
DEFAULT_ENCODED
parser if the input has not already had %xx
sequences
decoded.
Let's build upon the previous example and parse the generated URL-encoded string back into the original bean.
We start with the URL-encoded string that was generated.
// Use serializer with readable output.
UonSerializer s = UonSerializer.DEFAULT_READABLE ;
// Create our bean.
Person p = new Person(1, "John Smith" , "http://sample/addressBook/person/1" ,
"http://sample/addressBook" , "Aug 12, 1946" );
Address a = new Address();
a.uri = new URI("http://sample/addressBook/address/1" );
a.personUri = new URI("http://sample/addressBook/person/1" );
a.id = 1;
a.street = "100 Main Street" ;
a.city = "Anywhereville" ;
a.state = "NY" ;
a.zip = 12345;
a.isCurrent = true ;
p.addresses .add(a);
// Serialize the bean.
String urlencoded = s.serialize(p);
This code produced the following:
(
id =1 ,
name ='John+Smith' ,
uri =http://sample/addressBook/person/1 ,
addressBookUri =http://sample/addressBook ,
birthDate =1946-08-12T00:00:00Z ,
addresses =@(
(
uri =http://sample/addressBook/address/1 ,
personUri =http://sample/addressBook/person/1 ,
id =1 ,
street ='100+Main+Street' ,
city =Anywhereville ,
state =NY ,
zip =12345 ,
isCurrent =true
)
)
)
The code to convert this back into a bean is:
// Parse it back into a bean using the reusable JSON parser.
Person p = UonParser.DEFAULT .parse(urlencoded, Person.class );
// Render it back as JSON.
json = JsonSerializer.DEFAULT_LAX_READABLE .serialize(p);
We print it back out to JSON to show that all the data has been preserved:
{
id: 1 ,
name: 'John Smith' ,
uri: 'http://sample/addressBook/person/1' ,
addressBookUri: 'http://sample/addressBook' ,
birthDate: '1946-08-12T00:00:00Z' ,
addresses: [
{
uri: 'http://sample/addressBook/address/1' ,
personUri: 'http://sample/addressBook/person/1' ,
id: 1 ,
street: '100 Main Street' ,
city: 'Anywhereville' ,
state: 'NY' ,
zip: 12345 ,
isCurrent: true
}
]
}
3.1 - Parsing into generic POJO models
The URL-encoding parser is not limited to parsing back into the original bean classes.
If the bean classes are not available on the parsing side, the parser can also be used to parse into a
generic model consisting of Maps
, Collections
, and primitive objects.
You can parse into any Map
type (e.g. HashMap
, TreeMap
), but
using {@link org.apache.juneau.ObjectMap} is recommended since it has many convenience methods
for converting values to various types.
The same is true when parsing collections. You can use any Collection (e.g. HashSet
,
LinkedList
) or array (e.g. Object[]
, String[]
,
String[][]
), but using {@link org.apache.juneau.ObjectList} is recommended.
When the map or list type is not specified, or is the abstract Map
, Collection
,
or List
types, the parser will use ObjectMap
and ObjectList
by
default.
Starting back with our original URL-encoded string:
(
id =1 ,
name ='John+Smith' ,
uri =http://sample/addressBook/person/1 ,
addressBookUri =http://sample/addressBook ,
birthDate =1946-08-12T00:00:00Z ,
addresses =@(
(
uri =http://sample/addressBook/address/1 ,
personUri =http://sample/addressBook/person/1 ,
id =1 ,
street ='100+Main+Street' ,
city =Anywhereville ,
state =NY ,
zip =12345 ,
isCurrent =true
)
)
)
We can parse this into a generic ObjectMap
:
// Parse URL-encoded string into a generic POJO model.
ObjectMap m = UonParser.DEFAULT .parse(urlencoded, ObjectMap.class );
// Convert it back to JSON.
String json = JsonSerializer.DEFAULT_LAX_READABLE .serialize(m);
What we end up with is the exact same output.
Even the numbers and booleans are preserved because they are parsed into Number
and
Boolean
objects when parsing into generic models.
{
id: 1 ,
name: 'John Smith' ,
uri: 'http://sample/addressBook/person/1' ,
addressBookUri: 'http://sample/addressBook' ,
birthDate: '1946-08-12T00:00:00Z' ,
addresses: [
{
uri: 'http://sample/addressBook/address/1' ,
personUri: 'http://sample/addressBook/person/1' ,
id: 1 ,
street: '100 Main Street' ,
city: 'Anywhereville' ,
state: 'NY' ,
zip: 12345 ,
isCurrent: true
}
]
}
Once parsed into a generic model, various convenience methods are provided on the ObjectMap
and ObjectList
classes to retrieve values:
// Parse URL-encoded string into a generic POJO model.
ObjectMap m = UonParser.DEFAULT .parse(urlencoded, ObjectMap.class );
// Get some simple values.
String name = m.getString("name" );
int id = m.getInt("id" );
// Get a value convertable from a String.
URI uri = m.get(URI.class , "uri" );
// Get a value using a transform.
CalendarSwap transform = new CalendarSwap.ISO8601DTZ();
Calendar birthDate = m.get(transform, "birthDate" );
// Get the addresses.
ObjectList addresses = m.getObjectList("addresses" );
// Get the first address and convert it to a bean.
Address address = addresses.get(Address.class , 0);
As a general rule, parsing into beans is often more efficient than parsing into generic models.
And working with beans is often less error prone than working with generic models.
3.2 - Configurable properties
See the following classes for all configurable properties that can be used on this parser:
-
{@link org.apache.juneau.BeanContext} - Bean context properties.
-
{@link org.apache.juneau.uon.UonParserContext} - UON parser context properties.
-
{@link org.apache.juneau.urlencoding.UrlEncodingParserContext} - URL-Encoding parser context properties.
3.3 - Other notes
-
Like all other Juneau parsers, the URL-encoding parsers are thread safe and maintain an internal cache
of bean classes encountered.
For performance reasons, it's recommended that parser be reused whenever possible instead of always
creating new instances.
*** fín ***