Layouts
Common concerns
This section introduces you to some common concerns shared by almost all predefined layouts that you need to be aware of while using them.
Structured logging
In almost any modern production deployment, logs are no more written to files read by engineers while troubleshooting, but forwarded to log ingestion systems (Elasticsearch, Google Cloud Logging, etc.) for several observability use cases ranging from logging to metrics. This necessitates the applications to structure their logs in a machine-readable way ready to be delivered to an external system. This act of encoding logs following a certain structure is called structured logging.
Log4j strives to provide top of the class support for structured logging. To create an end-to-end experience, it provides several structured message types along with layouts supporting structured logging.
We recommend JSON Template Layout for structured logging purposes.
Character encoding
All predefined layouts produce String
that eventually get converted into a byte
using the Charset
configured.
While doing so, unless an explicit encoding configuration is stated, they use UTF-8
by default.
If you want all your log events to be formatted in a certain character encoding that is different from what the employed layout defaults to, make sure to configure the layout’s character encoding as needed.
Location information
Several layouts offer directives to include the location information: the caller class, method, file, and line. Log4j takes a snapshot of the stack, and walks the stack trace to find the location information. This is an expensive operation and should be avoided in performance-sensitive setups.
Note that the caller class of the location information and the logger name are two different things. In most setups just using the logger name – which doesn’t incur any overhead to obtain while logging! – is a sufficient and zero-cost substitute for the caller class. Example demonstrating that the logger name can be a substitute for the caller name
In the above example, if the caller class (which is expensive to compute!) is omitted in the layout, the produced log line will still be likely to contain sufficient information to trace back the source by just looking at the logger name. |
Asynchronous loggers need to capture the location information before passing the log message to another thread; otherwise the location information will be lost after that point. Due to associated performance implications, asynchronous loggers and asynchronous appenders do not include location information by default. You can override the default behaviour in your asynchronous logger or asynchronous appender configuration.
Even if a layout is configured not to request location information, it might use it if the information is already available. This is always the case if the location information is captured at build time using the Log4j Transform Maven Plugin. |
Collection
Log4j bundles predefined layouts to assist in several common deployment use cases. Let’s start with shortcuts to most used ones:
-
Are you looking for a production-grade JSON layout ready to be deployed to a log ingestion system such as Elasticsearch or Google Cloud? Refer to JSON Template Layout.
-
Are you looking for a layout that encodes log events in a human-readable format suitable for tests and local development? Refer to Pattern Layout.
Following sections explain all predefined layouts in detail.
CSV Layouts
There are two layouts performing Comma Separated Value (CSV) encoding:
CSV Parameter Layout
CsvParameterLayout
encodes only the parameters of the message of a log event.
Generated CSV records will be composed of fields denoting the message parameters.
Click here for examples
Given the following statement
LOGGER.info("Record1 {} {}", "arg1", "arg2");
LOGGER.error("Record2 {} {} {}", "arg3", "arg4", "arg5", throwable);
CsvParameterLayout
will output
arg1,arg2
arg3,arg4,arg5
The same can be achieved using ObjectArrayMessage
as follows:
LOGGER.info(new ObjectArrayMessage("arg1", "arg2"));
LOGGER.info(new ObjectArrayMessage("arg3", "arg4", "arg5"));
CSV Log Event Layout
CsvLogEventLayout
encodes the complete log event, including the formatted message.
Generated CSV records will be composed of following fields in the given order:
-
Time (in nanoseconds)
-
Time (in milliseconds)
-
Level
-
Thread ID
-
Thread name
-
Thread priority
-
Message (formatted, hence including parameters)
-
Logger FQCN
-
Logger name
-
Marker
-
Throwable
-
Source
-
Thread context map
-
Thread context stack
Click here for examples
Given the following statement
LOGGER.debug("one={}, two={}, three={}", 1, 2, 3);
CsvLogEventLayout
will output
0,1441617184044,DEBUG,main,"one=1, two=2, three=3",org.apache.logging.log4j.spi.AbstractLogger,,,,org.apache.logging.log4j.core.layout.CsvLogEventLayoutTest.testLayout(CsvLogEventLayoutTest.java:98),{},[]
Configuration
Both CsvParameterLayout
and CsvLogEventLayout
are configured with the following parameters:
Parameter | Type | Description |
---|---|---|
|
|
A predefined format name ( |
|
|
The field delimiter character |
|
|
The escape character |
|
|
The quote character |
|
|
A quote mode name ( |
|
|
The string to denote |
|
|
The record separator string |
|
|
The character encoding |
|
|
The header to include when the stream is opened |
|
|
The footer to include when the stream is closed |
Additional runtime dependencies are required for using CSV layouts:
-
Maven
-
Gradle
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-csv</artifactId>
<version>1.12.0</version>
<scope>runtime</scope>
</dependency>
runtimeOnly 'org.apache.commons:commons-csv:1.12.0'
GELF Layout
GelfLayout
encodes log events in the GELF specification version 1.1
.
It can compress the output when it exceeds a certain threshold.
This layout does not implement chunking.
This layout is planned to be removed in the next major release! Unless compression is needed, we advise you to use JSON Template Layout instead, which provides GELF Layout support out of the box and offers more capabilities and performance. |
GELF Layout is configured with the following parameters:
Parameter | Type | Description |
---|---|---|
|
|
|
|
|
Triggers compression if the output is larger than this number of bytes (optional, defaults to 1024) |
|
|
The compression to use.
It is optional, and defaults to |
|
|
The value of the |
|
|
Whether to include fields from |
|
|
Whether to include |
|
|
Whether to include full stacktrace of logged |
|
|
Whether to include thread context as additional fields (optional, defaults to |
|
|
A comma separated list of attributes from the |
|
|
A comma separated list of attributes from the |
|
|
A string to prepend to all elements of the |
|
The pattern to use to format the |
|
|
|
If |
The |
||
|
|
A comma separated list of Thread Context map keys to exclude when formatting the event.
This attribute only applies when |
|
|
A comma separated list of Thread Context map keys to include when formatting the event.
This attribute only applies when |
|
|
A string to prepend to all elements of the Thread Context map when rendered as a field (defaults to an empty string) |
GELF Layout is garbage-free if compressionType
is OFF
and there are no additional fields containing ${
.
Example configurations
-
XML
-
JSON
-
YAML
-
Properties
log4j2.xml
<Console name="CONSOLE">(1)
<GelfLayout messagePattern="%d %5p [%t] %c{1} %X{loginId, requestId} - %m%n"
includeThreadContext="true"
threadContextIncludes="loginId,requestId">
<KeyValuePair key="additionalField1" value="constant value"/>
<KeyValuePair key="additionalField2" value="$${ctx:key}"/>
</GelfLayout>
</Console>
<Socket name="GRAYLOG_TCP" protocol="TCP" host="graylog.domain.com" port="12201">(2)
<GelfLayout host="someserver" compressionType="OFF" includeNullDelimiter="true"/>
</Socket>
<Socket name="GRAYLOG_UDP" protocol="UDP" host="graylog.domain.com" port="12201">(3)
<GelfLayout host="someserver" compressionType="ZLIB" compressionThreshold="1024"/>
</Socket>
log4j2.json
"Console": {
"name": "CONSOLE",
"GelfLayout": { (1)
"messagePattern": "%d %5p [%t] %c{1} %X{loginId, requestId} - %m%n",
"includeThreadContext": "true",
"threadContextIncludes": "loginId,requestId",
"KeyValuePair": [
{
"key": "additionalField1",
"value": "constant value"
},
{
"key": "additionalField2",
"value": "$${ctx:key}"
}
]
}
},
"SocketTcp": { (2)
"type": "Socket",
"name": "GRAYLOG_TCP",
"protocol": "TCP",
"host": "graylog.domain.com",
"port": 12201,
"GelfLayout": {
"host": "someserver",
"compressionType": "OFF",
"includeNullDelimiter": "true"
}
},
"SocketUdp": { (3)
"type": "Socket",
"name": "GRAYLOG_UDP",
"protocol": "UDP",
"host": "graylog.domain.com",
"port": 12201,
"GelfLayout": {
"host": "someserver",
"compressionType": "ZLIB",
"compressionThreshold": 1024
}
}
log4j2.yaml
Console:
name: "CONSOLE"
GelfLayout: (1)
messagePattern: "%d %5p [%t] %c{1} %X{loginId, requestId} - %m%n"
includeThreadContext: "true"
threadContextIncludes: "loginId,requestId"
KeyValuePair:
- key: "additionalField1"
value: "constant value"
- key: "additionalField2"
value: "$${ctx:key}"
SocketTcp: (2)
type: "Socket"
name: "GRAYLOG_TCP"
protocol: "TCP"
host: "graylog.domain.com"
port: 12201
GelfLayout:
host: "someserver"
compressionType: "OFF"
includeNullDelimiter: "true"
SocketUdp: (3)
type: "Socket"
name: "GRAYLOG_UDP"
protocol: "UDP"
host: "graylog.domain.com"
port: 12201
GelfLayout:
host: "someserver"
compressionType: "ZLIB"
compressionThreshold: 1024
log4j2.properties
appender.0.type = Console (1)
appender.0.name = CONSOLE
appender.0.layout.type = GelfLayout
appender.0.layout.messagePattern = %d %5p [%t] %c{1} %X{loginId, requestId} - %m%n
appender.0.layout.includeThreadContext = true
appender.0.layout.threadContextIncludes = true
appender.0.layout.KeyValuePair.0.type = KeyValuePair
appender.0.layout.KeyValuePair.0.key = additionalField1
appender.0.layout.KeyValuePair.0.value = constant value
appender.0.layout.KeyValuePair.1.type = KeyValuePair
appender.0.layout.KeyValuePair.1.key = additionalField2
appender.0.layout.KeyValuePair.1.value = $${ctx:key}
appender.1.type = Socket (2)
appender.1.name = GRAYLOG_TCP
appender.1.protocol = TCP
appender.1.host = graylog.domain.com
appender.1.port = 12201
appender.1.layout.type = GelfLayout
appender.1.layout.host = someserver
appender.1.layout.compressionType = OFF
appender.1.layout.includeNullDelimiter = true
appender.2.type = Socket (3)
appender.2.name = GRAYLOG_UDP
appender.2.protocol = UDP
appender.2.host = graylog.domain.com
appender.2.port = 12201
appender.2.layout.type = GelfLayout
appender.2.layout.host = someserver
appender.2.layout.compressionType = ZLIB
appender.2.layout.compressionThreshold = 1024
1 | Configuration with additional key value pairs |
2 | Configuration for appending to a Graylog server using TCP |
3 | Configuration for appending to a Graylog server using UDP |
HTML Layout
HtmlLayout
generates an HTML page, and adds each log event to a row in a table.
It is configured with the following parameters:
Parameter | Type | Description |
---|---|---|
|
|
The character encoding |
|
|
The |
|
|
The date format of the log event.
The default is |
|
|
The |
|
|
The |
|
|
If |
|
|
The time zone ID of the log event.
If not specified, this layout uses the |
|
|
The HTML page title |
JSON Layout
JsonLayout
encodes a log event into JSON.
JSON Layout is considered deprecated. It is succeeded by JSON Template Layout providing more capabilities and efficiency. |
Click for an example output
{
"instant": {
"epochSecond": 1493121664,
"nanoOfSecond": 118000000
},
"thread": "main",
"level": "INFO",
"loggerName": "HelloWorld",
"marker": {
"name": "child",
"parents": [
{
"name": "parent",
"parents": [
{
"name": "grandparent"
}
]
}
]
},
"message": "Hello, world!",
"thrown": {
"commonElementCount": 0,
"message": "error message",
"name": "java.lang.RuntimeException",
"extendedStackTrace": [
{
"class": "logtest.Main",
"method": "main",
"file": "Main.java",
"line": 29,
"exact": true,
"location": "classes/",
"version": "?"
}
]
},
"contextStack": ["one", "two"],
"endOfBatch": false,
"loggerFqcn": "org.apache.logging.log4j.spi.AbstractLogger",
"contextMap": {
"bar": "BAR",
"foo": "FOO"
},
"threadId": 1,
"threadPriority": 5,
"source": {
"class": "logtest.Main",
"method": "main",
"file": "Main.java",
"line": 29
}
}
JSON Layout is configured with the following parameters:
Parameter Name | Type | Description |
---|---|---|
|
|
|
|
|
The character encoding (defaults to |
|
|
If |
|
|
If |
|
|
If set, overrides the default end-of-line string.
For instance, set it to |
|
|
If |
|
|
The footer to include when the stream is closed and |
|
|
The header to include when the stream is opened and |
|
|
If |
|
|
If |
|
|
If |
|
|
If |
|
|
If |
|
|
If |
|
|
If |
|
|
If |
Additional runtime dependencies are required for using JSON Layout:
-
Maven
-
Gradle
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.18.2</version>
<scope>runtime</scope>
</dependency>
runtimeOnly 'com.fasterxml.jackson.core:jackson-databind:2.18.2'
JSON Template Layout
JsonTemplateLayout
is a customizable, efficient, and garbage-free JSON generating layout.
It encodes LogEvent
s according to the structure described by the JSON template provided.
For instance, given the following event template stored in MyLayout.json
in your classpath:
{
"instant": { (1)
"$resolver": "timestamp",
"pattern": {
"format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
"timeZone": "UTC"
}
},
"someConstant": 1, (2)
"message": { (3)
"$resolver": "message",
"stringified": true
}
}
1 | Using the timestamp event template resolver to populate the instant field |
2 | Passing a constant that will be rendered as is |
3 | Using the message event template resolver to populate the message field |
in combination with the below layout configuration:
-
XML
-
JSON
-
YAML
-
Properties
log4j2.xml
<JsonTemplateLayout eventTemplateUri="classpath:MyLayout.json"/>
log4j2.json
"JsonTemplateLayout": {
"eventTemplateUri": "classpath:MyLayout.json"
}
log4j2.yaml
JsonTemplateLayout:
eventTemplateUri: "classpath:MyLayout.json"
log4j2.properties
appender.0.layout.type = JsonTemplateLayout
appender.0.layout.eventTemplateUri = classpath:MyLayout.json
JSON Template Layout generates JSON as follows:
{"instant":"2017-05-25T19:56:23.370Z","someConstant":1,"message":"Hello, error!"} (1)
1 | JSON pretty-printing is not supported for performance reasons. |
Good news is JSON Template Layout is perfectly production-ready without any configuration! It bundles several predefined event templates modeling popular JSON-based log formats.
Read more on JSON Template Layout…
Message Layout
MessageLayout
is a special layout that extracts the
Message
contained in a log event.
It is currently only useful with the
JDBC Appender.
Pattern Layout
PatternLayout
is a customizable, efficient, garbage-free, and human-readable string generating layout using a user-provided pattern.
It is analogous to String#format()
with specialized directives on injecting certain properties of a LogEvent
.
Pattern Layout is not intended for structural logging purposes. For production environments, you are strongly advised to use JSON Template Layout producing JSON output ready to be delivered to log ingestion systems such as Elasticsearch or Google Cloud Logging. |
A conversion pattern is composed of literal text and format control expressions.
For instance, given the %-5p [%t]: %m%n
pattern, following statements
LOGGER.debug("Message 1");
LOGGER.warn("Message 2");
will yield the output
DEBUG [main]: Message 1
WARN [main]: Message 2
Read more on Pattern Layout…
RFC 5424 Layout
Rfc5424
Layout encodes log events according to the Syslog message format described in RFC 5424.
RFC 5424 obsoletes RFC 3164, implemented by Syslog Layout. |
RFC 5424 Layout is configured with the following parameters:
Parameter | Type | Description |
---|---|---|
|
|
The |
|
|
The character encoding (defaults to |
|
|
The |
|
|
An |
|
|
The name of |
|
|
The default Structured Data ID to use when formatting according to RFC 5424.
If the log event contains a |
|
|
Indicates whether data from the Thread Context map will be included in the RFC 5424 Syslog record (defaults to |
|
Allows arbitrary Thread Context map entries.
To use, include a |
|
|
|
A comma-separated list of Thread Context map (aka, MDC) keys that should be excluded.
This is mutually exclusive with |
|
|
A comma-separated list of Thread Context map (aka, MDC) keys that should be included.
Any keys in the thread context map not found in the list will be excluded.
This option is mutually exclusive with |
|
|
The ID to use for the Thread Context map (aka, MDC) Structured Data Element.
It defaults to |
|
|
A string to be prepended to each Thread Context map (aka, MDC) key to distinguish it from event attributes.
It defaults to |
|
|
A comma-separated list of Thread Context map (aka, MDC) keys that must be present.
If a key is not present, a
|
|
|
The default value to be used in the |
|
|
If |
|
|
The string that should be used to replace newlines within the message text |
RFC 5424 Layout has specialized handling for StructuredDataMessage
s.
By combining two, users can have complete control on how their message is encoded in a way compliant with RFC 5424, while RFC 5424 Layout will make sure the rest of the information attached to the log event is properly injected.
Serialized Layout
SerializedLayout
encodes a log event using Java Serialization.
It accepts no configuration parameters.
This layout has been deprecated since version |
xref:plugin-reference.adoc#org-apache-logging-log4j_log4j-core_org-apache-logging-log4j-core-layout-SerializedLayout[{plugin-reference-marker} Plugin reference for `SerializedLayout`]
Syslog Layout
SyslogLayout
encodes log events according to the syslog message format described in RFC 3164.
This matches the same format used by Log4j 1.
RFC 3164, implemented by Syslog Layout, is obsoleted by RFC 5424, implemented by RFC 5424 Layout. |
Syslog Layout is configured with the following parameters:
Parameter | Type | Description |
---|---|---|
|
|
The character encoding (defaults to |
|
|
The name of |
|
|
If |
|
|
The string that should be used to replace newlines within the message text |
XML Layout
XmlLayout
encodes a log event into XML.
This layout is planned to be removed in the next major release! XML Layout users are strongly advised to migrate to another layout! |
Click for an example output
<Event xmlns="https://logging.apache.org/log4j/2.0/events"
level="INFO"
loggerName="HelloWorld"
endOfBatch="false"
thread="main"
loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger"
threadId="1"
threadPriority="5">
<Instant epochSecond="1493121664" nanoOfSecond="118000000"/>
<Marker name="child">
<Parents>
<Marker name="parent">
<Parents>
<Marker name="grandparent"/>
</Parents>
</Marker>
</Parents>
</Marker>
<Message>Hello, world!</Message>
<ContextMap>
<item key="bar" value="BAR"/>
<item key="foo" value="FOO"/>
</ContextMap>
<ContextStack>
<ContextStackItem>one</ContextStackItem>
<ContextStackItem>two</ContextStackItem>
</ContextStack>
<Source
class="logtest.Main"
method="main"
file="Main.java"
line="29"/>
<Thrown commonElementCount="0" message="error message" name="java.lang.RuntimeException">
<ExtendedStackTrace>
<ExtendedStackTraceItem
class="logtest.Main"
method="main"
file="Main.java"
line="29"
exact="true"
location="classes/"
version="?"/>
</ExtendedStackTrace>
</Thrown>
</Event>
XML Layout configuration is identical to the one of JSON Layout, with following exceptions:
-
The
header
andfooter
parameters are discarded -
If
complete=true
, each log event will be encoded into a well-formed XML document, such that:-
<?xml version…
preamble will be added -
The root
<event
element will be enriched withnamespace="http://logging.apache.org/log4j/2.0/events"
attribute
-
Additional runtime dependencies are required for using XML Layout:
-
Maven
-
Gradle
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
<version>2.18.2</version>
<scope>runtime</scope>
</dependency>
runtimeOnly 'com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.18.2'
YAML Layout
YamlLayout
encodes a log event into YAML.
This layout is planned to be removed in the next major release! YAML is a superset of JSON. We strongly advise existing YAML Layout users to migrate to JSON Template Layout providing more capabilities and efficiency. |
Click for an example output
instant:
epochSecond: 1493121664
nanoOfSecond: 118000000
thread: "main"
level: "INFO"
loggerName: "HelloWorld"
marker:
name: "child"
parents:
- name: "parent"
parents:
- name: "grandparent"
message: "Hello, world!"
thrown:
commonElementCount: 0
message: "error message"
name: "java.lang.RuntimeException"
extendedStackTrace:
- class: "logtest.Main"
method: "main"
file: "Main.java"
line: 29
exact: true
location: "classes/"
version: "?"
contextStack:
- "one"
- "two"
endOfBatch: false
loggerFqcn: "org.apache.logging.log4j.spi.AbstractLogger"
contextMap:
bar: "BAR"
foo: "FOO"
threadId: 1
threadPriority: 5
source:
class: "logtest.Main"
method: "main"
file: "Main.java"
line: 29
YAML Layout configuration is identical to the one of JSON Layout, with following exceptions:
-
The
header
defaults to an empty string -
The
footer
defaults to an empty string
Additional runtime dependencies are required for using XML Layout:
-
Maven
-
Gradle
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-yaml</artifactId>
<version>2.18.2</version>
<scope>runtime</scope>
</dependency>
runtimeOnly 'com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.18.2'
Extending
Layouts are plugins implementing the Layout
interface.
This section will guide you on how to create custom ones.
While the predefined layout collection should address most common use cases, you might find yourself needing to implement a custom one. If this is the case, we really appreciate it if you can share your use case in a user support channel. |
Plugin preliminaries
Log4j plugin system is the de facto extension mechanism embraced by various Log4j components. Plugins provide extension points to components, that can be used to implement new features, without modifying the original component. It is analogous to a dependency injection framework, but curated for Log4j-specific needs.
In a nutshell, you annotate your classes with @Plugin
and their (static
) factory methods with @PluginFactory
.
Last, you inform the Log4j plugin system to discover these custom classes.
This is done using running the PluginProcessor
annotation processor while building your project.
Refer to Plugins for details.
Extending layouts
Layouts are plugins implementing the Layout
interface.
If your layout is a String
-based one, we recommend you to extend your plugin class from AbstractStringLayout
, which contains convenience for some of the boilerplate code shared by String
-based layouts.
While annotating your layout with @Plugin
, you need to make sure that
-
It has a unique
name
attribute across all availableLayout
plugins -
The
category
attribute is set toNode.CATEGORY
You can check out following files for examples:
-
SyslogLayout.java
– simple, single-file, extending fromAbstractStringLayout
-
JsonTemplateLayout.java
– advanced, using plugins for composing several of its features, contains recycler concept for garbage-free operation, extends fromStringLayout