Rolling file appenders

Log4j Core provides multiple appenders that allow to archive the current log file and truncate it.

The rolling file appenders support many configuration options.

If you are just interested in some configuration examples, see the Configuration recipes section.

Appenders

Log4j Core provides two rolling file appenders:

RollingFile: The RollingFile Appender uses FileOutputStream to access log files.
RollingRandomAccessFile: The RollingRandomAccessFile Appender uses RandomAccessFile to access log files.

Two appenders, even from different logger contexts, share a common RollingFileManager if:

they have the same non-null value of the fileName attribute.
they have no fileName attribute, but have the same value of the filePattern attribute.

Sharing a RollingFileManager guarantees that rollovers will occur only once, but requires most of the remaining configuration parameters to be the same.

Common configuration

Attribute Type Default value Description

Required

filePattern

Path

The pattern for the archived log files. See Conversion patterns for details

If fileName is null, the file pattern will also be used for the current file.

name

String

The name of the appender.

Optional

bufferSize

int

8192

The size of the ByteBuffer internally used by the appender.

See Buffering for more details.

bufferedIo

boolean

true

If set to true, Log4j Core will format each log event in an internal buffer, before sending it to the underlying resource.

The RandomAccessRollingFile Appender always enables the internal buffer.

See Buffering for more details.

createOnDemand

boolean

false

The appender creates the file on-demand. The appender only creates the file when a log event passes all filters and is routed to this appender. Defaults to false.

fileName

Path

null

The path to the current log file, can be null. If the folder containing the file does not exist, it will be created.

This attribute should not contain any runtime lookups nor pattern converters.

To use pattern converters, see filePattern.

filePermissions

PosixFilePermissions

null

If not null, it specifies the POSIX file permissions to apply to each created file. The permissions must be provided in the format used by PosixFilePermissions.fromString(), e.g. rw-rw----.

The underlying files system shall support POSIX file attribute view.

fileOwner

String

null

If not null, it specifies the file owner to apply to each created file.

The underlying files system shall support file owner attribute view.

fileGroup

String

null

If not null, it specifies the file group owner to apply to each created file.

The underlying files system shall support POSIX file attribute view.

ignoreExceptions

boolean

true

If false, logging exception will be forwarded to the caller of the logging statement. Otherwise, they will be ignored.

Logging exceptions are always also logged to Status Logger

immediateFlush

boolean

true

If set to true, the appender will flush its internal buffer after each event.

See Buffering for more details.

Table 2. Common nested elements
Type	Multiplicity	Description
`Filter`	zero or one	Allows filtering log events just before they are formatted and sent. See also appender filtering stage.
`Layout`	zero or one	Formats log events. See Layouts for more information.
`TriggeringPolicy`	one	Determines when to archive the current log file. See Triggering Policies for more information.
`RolloverStrategy`	zero or one	Determines the actions performed during a rollover. See Rollover strategies for more information.

Conversion patterns

The filePattern attribute accepts the following pattern specifiers:

runtime lookups, which are evaluated in a global context.
a %d{...} pattern, functionally identical to the homonymous PatternLayout pattern except it uses the actual or inferred timestamp of the previous rollover.
a %i pattern that expands to the computed index for the archived file.

All patterns accept also format specifiers, e.g. %03i prints the index as zero-padded number with 3 digits.

The $${date:...} runtime lookup and %d{...} conversion pattern are not equivalent:

$${date:...} formats the current date.
%d{...} formats the date of the previous rollover.

Setting the filePattern property to

$${date:yyyy-MM}/%d{yyyy-MM-dd}.log

will write the 2024-06-30.log log file into the 2024-07 directory. Consider using

%d{yyyy-MM}/%d{yyyy-MM-dd}.log

instead.

`RollingFile` configuration

The RollingFile Appender provides the following configuration options, beyond the common ones:

Table 3. `RollingFile` configuration attributes
Attribute	Type	Default value	Description
`append`	`boolean`	`true`	If `true`, the log file will be opened in `APPEND` mode. On most systems this guarantees atomic writes to the end of the file, even if the file is opened by multiple applications.
`locking`	`boolean`	`false`	If `true`, Log4j will lock the log file at each log event. Note that the effects of this setting depend on the Operating System: some systems like most POSIX OSes do not offer mandatory locking, but only advisory file locking. This setting can also reduce the performance of the appender.

📖 Plugin reference for RollingFile

`RollingRandomAccessFile` configuration

The RollingRandomAccessFile Appender provides the following configuration options, beyond the common ones:

Table 4. `RollingRandomAccessFile` configuration attributes
Attribute	Type	Default value	Description
`append`	`boolean`	`true`	If `true`, the appender starts writing at the end of the file. This setting does not give the same atomicity guarantees as for the `RollingFile` Appender. The log file cannot be opened by multiple applications at the same time.

📖 Plugin reference for RollingRandomAccessFile

Triggering Policies

Triggering policies are Log4j plugins that implement the TriggeringPolicy interface and are used to decide when is it time to rollover the current log file.

📖 Plugin reference for TriggeringPolicy

Common concerns

Triggering policies usually decide to rollover a file based on two parameters:

the size of the current file.

While all JVMs can reliably check the size of a file, Log4j only checks it at startup and at each rollover and infers all file size changes, based on the size of logs delivered. If multiple managers are writing to the same file, the size computation will be off.

the creation timestamp of the current file.

Not all file systems support a creation timestamp. Most notably, POSIX does not specify such a timestamp, so the value used by Log4j might depend on the type of filesystem used and the JVM. If the creation timestamp is not available, the last modification timestamp is used, which might differ by up to a whole rollover period.

See BasicFileAttributes.creationTime() for more details.

After the first rollover Log4j uses the timestamp of the rollover as creation timestamp.

`OnStartupTriggeringPolicy`

The OnStartupTriggeringPolicy policy causes a rollover only once during the lifetime of a JVM. A rollover will occur if the log file was created before the start time of the current JVM and the minimum file size is met or exceeded.

Table 5. `OnStartupTriggeringPolicy` configuration attributes
Attribute	Type	Default value	Description
`minSize`	`long`	`1`	The minimum size the file must have to roll over.

The OnStartupTriggeringPolicy uses JMX to retrieve the start time of the JVM.

On platforms where JMX is absent or disabled, like Android or Google App Engine, the OnStartupTriggeringPolicy will use the timestamp, when its class was loaded instead.

📖 Plugin reference for OnStartupTriggeringPolicy

`CronTriggeringPolicy`

The CronTriggeringPolicy triggers rollover based on a CRON expression.

The filePattern attribute of the Appender should contain a %d{...} timestamp, otherwise the target file will be overwritten on each rollover.

This policy is controlled by a timer and is asynchronous in processing log events, so it is possible that log events from the previous or next period may appear at the beginning or end of the log file.

Table 6. `CronTriggeringPolicy` configuration attributes
Attribute	Type	Default value	Description
`evaluateOnStartup`	boolean	`false`	On startup the cron expression will be evaluated against the current file’s creation timestamp. If a rollover should have occurred between that time and the current time the file will be immediately rolled over.
`schedule`	`CronExpression`	`0 0 0 * * ?`	The cron expression, using the same syntax as the Quartz scheduler. The supported fields are in order: second minute hour day of month month day of week See `CronExpression` for a full specification of the format.

📖 Plugin reference for CronTriggeringPolicy

`SizeBasedTriggeringPolicy`

The SizeBasedTriggeringPolicy causes a rollover once the file has reached the specified size.

The size can be specified in bytes, with the suffix KB, MB, GB, or TB for example 20MB. size. The size may also contain a fractional value such as 1.5 MB. The size is evaluated using the Java root Locale so a period must always be used for the fractional unit.

When combined with a time-based triggering policy, the filePattern attribute of the Appender should contain an %i conversion pattern. Otherwise, the target file will be overwritten on each rollover.

Table 7. `SizeBasedTriggeringPolicy` configuration attributes
Attribute	Type	Default value	Description
`size`	`FileSize`	`10 MB`	The maximum file size of the current log file. Once this size is reached, a rollover will occur. The syntax of this attribute is: <number> [ " " ] [ <unit> ] i.e.: a number recognized by `NumberFormat` followed by optional whitespace followed by an optional unit: `k`, `kB`, `M`, `MB`, `G`, `GB`, `T`, `TB`.

📖 Plugin reference for SizeBasedTriggeringPolicy

`TimeBasedTriggeringPolicy`

The TimeBasedTriggeringPolicy causes a rollover, when the smallest time unit in filePattern changes value. So the rollover can occur at the end of a month, week, at midnight, end of the hour, etc.

The filePattern attribute of the Appender should contain a %d{...} timestamp, otherwise the target file will be overwritten on each rollover.

Multiple %d patterns can be used. The rollover frequency time unit will be determined by the last %d pattern.

Table 8. `TimeBasedTriggeringPolicy` configuration attributes
Attribute	Type	Default value	Description
`interval`	`int`	`1`	How often a rollover should occur based on the most specific time unit in the last `%d{...}` pattern.
`modulate`	`boolean`	`false`	If `true`, the rollover will be aligned to a boundary of `interval` time units. For example, if the rotation frequency is hourly with an interval of `4` and the application starts at `3:14`: `false` Will roll over at `3:00`, `7:00`, `11:00`, `15:00`, `19:00` and `23:00` each day. `true` Will roll over at `0:00`, `4:00`, `8:00`, `12:00`, `16:00` and `20:00` each day. Only applies if `interval` is greater than `1`.
`maxRandomDelay`	`int`	`0`	Indicates the maximum number of seconds to randomly delay a rollover. This setting is useful on servers where multiple applications are configured to roll over log files at the same time and can spread the load of doing so across time.

When the rollover frequency is daily or lower, the rolling file appender will rotate files at midnight of the server’s default timezone.

You can modify the time of the rollover by specifying a different timezone in the %d pattern. See date converter syntax for more details.

📖 Plugin reference for TimeBasedTriggeringPolicy

Multiple policies

A rolling file appender allows only one nested triggering policy element. If you wish to use multiple policies, you need to wrap them in a Policies element. The element itself has no configuration attributes.

The effects of using both time-based triggering policies (CronTriggeringPolicy and TimeBasedTriggeringPolicy) at the same time are undefined.

For example, the following XML fragment defines policies that rollover the log:

when the JVM starts.
when the log size reaches 10 MB.
at midnight local time.

XML
JSON
YAML
Properties

Snippet from an example log4j2.xml

<RollingFile name="FILE"
             fileName="app.log"
             filePattern="app.%d{yyyy-MM-dd}.%i.log">
  <JsonTemplateLayout/>
  <Policies>
    <OnStartupTriggeringPolicy/>
    <SizeBasedTriggeringPolicy/>
    <TimeBasedTriggeringPolicy/>
  </Policies>
</RollingFile>

Snippet from an example log4j2.json

"RollingFile": {
  "name": "FILE",
  "fileName": "app.log",
  "filePattern": "app.%d{yyyy-MM-dd}.%i.log",
  "JsonTemplateLayout": {},
  "Policies": {
    "OnStartupTriggeringPolicy": {},
    "SizeBasedTriggeringPolicy": {},
    "TimeBasedTriggeringPolicy": {}
  }
}

Snippet from an example log4j2.yaml

RollingFile:
  name: "FILE"
  fileName: "app.log"
  filePattern: "app.%d{yyyy-MM-dd}.%i.log"
  JsonTemplateLayout: {}
  Policies:
    OnStartupTriggeringPolicy: {}
    SizeBasedTriggeringPolicy: {}
    TimeBasedTriggeringPolicy: {}

Snippet from an example log4j2.properties

appender.0.type = RollingFile
appender.0.name = FILE
appender.0.fileName = app.log
appender.0.filePattern = app.%d{yyyy-MM-dd}.%i.log

appender.0.layout.type = JsonTemplateLayout

appender.0.policy.type = Policies
appender.0.policy.0.type = OnStartupTriggeringPolicy
appender.0.policy.1.type = SizeBasedTriggeringPolicy
appender.0.policy.2.type = TimeBasedTriggeringPolicy

📖 Plugin reference for Policies

Rollover strategies

A rollover strategy determines how old archive files are rotated or deleted to make place for newer ones. The actions performed during a rollover include:

Renaming files in a cyclic way. See Incrementing file indexes for more details.
Compression of archived log files. See Compressing archived files for more details.
Deletion of old archived log files. See Delete action for more details.
Changing the POSIX permissions of archived log files. See PosixViewAttribute action for more details.

There are two different rollover strategies available out of the box:

DefaultRolloverStrategy: This is the default strategy used if the fileName configuration attribute is specified. It stores the current log file in the location specified by fileName and the archived log files in the location specified by filePattern.
DirectWriteRolloverStrategy: This is the default strategy used if the fileName configuration attribute is null. It stores the current log file directly in its rolled over location specified by filePattern.

Using runtime lookups in the fileName configuration attribute is discouraged, since it will break the logic of the DefaultRolloverStrategy. Use the DirectWriteRolloverStrategy instead by omitting the fileName attribute in your configuration.

Common configuration

Both rollover strategies support the following configuration parameters:

Table 9. Common rollover strategy configuration attributes
Attribute	Type	Default value	Description
`compressionLevel`	`int`	`7`	Determine the compression level of archived log files. See Compressing archived files for more details.
`tempCompressedFilePattern`	`Path`		Temporary location to store compressed files. See Compressing archived files for more details.
`stopCustomActionsOnError`	`boolean`	`true`	If `true`, custom actions will be stopped if one of them fails.

Table 10. Common Rollover Strategy nested elements
Type	Multiplicity	Description
`Action`	zero or more	Additional actions to perform on a rollover beyond file rotation and compression. See Optional actions for more details.

`DefaultRolloverStrategy` configuration

The DefaultRolloverStrategy supports additional attributes that control the incrementation of file indexes.

Table 11. `DefaultRolloverStrategy` configuration attributes
Attribute	Type	Default value	Description
`fileIndex`	enumeration	`max`	Determines the strategy for determining the value of the `%i` conversion pattern for the current log file. The supported values are: `min` uses the value of the `min` attribute. `max` uses the first unused integer between the `min` and `max` attributes. `nomax` uses the first unused integer not lower than the value of the `min` attribute.
`min`	`int`	`1`	Minimum value for the `%i` conversion pattern.
`max`	`int`	`7`	Maximum value for the `%i` conversion pattern. This attribute is ignored if `fileIndex` is set to `nomax`.

📖 Plugin reference for DefaultRolloverStrategy

`DirectWriteRolloverStrategy` configuration

The DirectWriteRolloverStrategy has a reduced set of configuration options for incrementing the file index:

the minimum file index is always 1.
the incrementing strategy is always max.
the maximum file index can be configured using the following configuration attribute:

Table 12. DirectWriteRolloverStrategy configuration attributes

Attribute Type Default value Description

maxFiles

int

7

Maximum value for the %i conversion pattern.

Table 12. `DirectWriteRolloverStrategy` configuration attributes
Attribute	Type	Default value	Description
`maxFiles`	`int`	`7`	Maximum value for the `%i` conversion pattern.

📖 Plugin reference for DirectWriteRolloverStrategy

Incrementing file indexes

During a rollover event, the current log file is moved to the location determined by evaluating filePattern, using min as value for the %i conversion pattern. If an old log file is already present in that location, the rolling file appender:

Tries to increment the value of %i using the strategy specified by the fileIndex configuration attribute.
If that fails, it removes the oldest log file and rotates the remaining ones to make place for a new log archive.

There are three strategies available:

min

Using the min strategy, the newest log file will have index min and the oldest one will have index max.

This is the log rotation strategy used by traditional UNIX tools and by Log4j 1. It is not the default strategy of Log4j 2.

Assuming min="1" and max="3" the rotation of the log files is represented in the graph below:

max

Using the max strategy, the oldest log file will have index min and the newest one will have index max.

This is the default strategy since Log4j 2.

Assuming min="1" and max="3" the rotation of the log files is represented in the graph below:

nomax: Using the nomax strategy no files will ever be deleted and newer archive files will be assigned increasing index numbers, starting from min.

Compressing archived files

When the current log file is archived, the rolling file appender can compress it. Compression is activated, based on the extension of the archived file name. The following extensions are recognized:

Extension Supports compressionLevel Description

Extension	Supports `compressionLevel`	Description
`.zip`	✓	ZIP archive using the DEFLATE algorithm
`.gz`	✓	GZIP archive using the DEFLATE algorithm
`.bz2`^dep	✗	BZip2 algorithm
`.deflate`^dep	✗	DEFLATE algorithm
`.pack200`^dep	✗	Pack200 algorithm
`.xz` ^dep	✗	XZ algorithm
`.zst` ^dep	✗	ZStandard algorithm

.zip

✓

ZIP archive using the DEFLATE algorithm

.gz

✓

GZIP archive using the DEFLATE algorithm

.bz2^dep

✗

BZip2 algorithm

.deflate^dep

✗

DEFLATE algorithm

.pack200^dep

✗

Pack200 algorithm

.xz ^dep

✗

XZ algorithm

.zst ^dep

✗

ZStandard algorithm

If the tempCompressedFilePattern attribute is set, the current log file:

will be compressed and stored in the location given by tempCompressedFilePattern
and then it will be moved to the location given by filePattern.

^dep

Additional dependencies are required to use these compression algorithms:

Maven
Gradle

<dependency>
  <groupId>org.apache.commons</groupId>
  <artifactId>commons-compress</artifactId>
  <version>1.27.1</version>
  <scope>runtime</scope>
</dependency>

runtimeOnly 'org.apache.commons:commons-compress:1.27.1'

The .xz and .zst extensions require additional dependencies. See Commons Compress documentation for more details.

Optional actions

The rotation of files and compression is always handled automatically by the rolling file appenders. Since Log4j 2.6, additional actions can be configured manually.

Log4j Core provides out-of-the-box two actions:

the Delete action to delete old log files after a rollover,
the PosixViewAttribute action to change the POSIX permissions of old log files.

📖 Plugin reference for AbstractPathAction

Common action configuration

Both actions support the following configuration attributes:

Attribute Type Default value Description

basePath

Path

It sets the base directory for the action. The action will be limited to files in this directory, and all paths will be relative to this directory.

Required

followLinks

boolean

false

If set to true, the action will follow symbolic links.

Setting this value to true will allow the action to access files outside basePath, which might introduce a security risk.

maxDepth

int

1

The maximum number of directory levels to visit. By default, the action does not recurse into subdirectories of basePath.

Table 14. Common action nested elements
Type	Multiplicity	Description
`PathCondition`	zero or more	A set of path conditions. The action will only be performed on files for which the condition returns `true`.

`Delete` action

The Delete Action deletes old log files that match the configured condition.

A limited version of this action is performed automatically to increment file indexes.

You only need an explicit Delete Action if you use a %d pattern.

Besides the common configuration options the Delete Action also supports the following options:

Table 15. `Delete` Action configuration attributes
Attribute	Type	Default value	Description
`testMode`	`boolean`	`false`	If `true`, no files will be deleted, but a Status Logger message of level `INFO` will be issued instead.

Table 16. `Delete` Action nested elements
Type	Multiplicity	Description
`PathCondition`	zero or more	If present, the action will only be performed on files for which the condition returns `true`. Required, unless a `ScriptCondition` is provided, in which case these elements are ignored.
`PathSorter`	zero or one	Provides a sorting order for files contained in `basePath`. The default implementation is `PathSortByModificationTime`, which sorts files by ascending modification timestamp.
`ScriptCondition`	zero or one	If present, all the nested `PathCondition`s are ignored and the provided script is executed to select the file to delete.

📖 Plugin reference for Delete

`PosixViewAttribute` action

This action allows modifying the POSIX attributes (owner, group and permissions) of archive log files.

By default, the POSIX attributes are inherited from the current log file.

This action is only necessary if archived log files must have different attributes.

Besides the common configuration options the PosixViewAttribute Action also supports the following options.

Table 17. `PosixViewAttribute` Action configuration attributes
Attribute	Type	Default value	Description
`filePermissions`	`PosixFilePermissions`	`null`	If not `null`, it specifies the POSIX file permissions to apply to each created file. The permissions must be provided in the format used by `PosixFilePermissions.fromString()`, e.g. `rw-rw----`. The underlying files system shall support POSIX file attribute view.
`fileOwner`	`String`	`null`	If not `null`, it specifies the file owner to apply to each created file. The underlying files system shall support file owner attribute view.
`fileGroup`	`String`	`null`	If not `null`, it specifies the file group owner to apply to each created file. The underlying files system shall support POSIX file attribute view.

Table 18. `PosixViewAttribute` Action nested elements
Type	Multiplicity	Description
`PathCondition`	one or more	A set of conditions to select the files to modify.

📖 Plugin reference for PosixViewAttribute

Path conditions

To select the files for additional actions, Log4j provides the following path conditions:

`IfAccumulatedFileSize`

When evaluated on a list of files, this condition sums the size of each file with the size of the preceding files. It returns true for files that exceed a configurable threshold.

The result of this condition depends on the sorting order on files. See PathSorter for more information.

Table 19. `IfAccumulatedFileSize` configuration attributes
Attribute	Type	Default value	Description
`exceeds`	`FileSize`		The threshold size for the condition to match. Admits the same syntax as the `size` attribute of `SizeBasedTriggeringPolicy`. Required

Table 20. `IfAccumulatedFileSize` nested elements
Type	Multiplicity	Description
`PathCondition`	zero or more	A set of optional nested conditions. This condition matches only if all the nested conditions also match.

📖 Plugin reference for IfAccumulatedFileSize

`IfAccumulatedFileCount`

When evaluated on a list of files, this condition returns true if the 1-based index of a file exceeds a configurable threshold.

The result of this condition depends on the sorting order on files. See PathSorter for more information.

Table 21. `IfAccumulatedFileCount` configuration attributes
Attribute	Type	Default value	Description
`exceeds`	`int`		The threshold for the condition to match. Required

Table 22. `IfAccumulatedFileCount` nested elements
Type	Multiplicity	Description
`PathCondition`	zero or more	A set of optional nested conditions. This condition matches only if all the nested conditions also match.

📖 Plugin reference for IfAccumulatedFileCount

`IfFileName`

Matches files based on their path relative to the base directory.

Table 23. `IfFileName` configuration attributes
Attribute	Type	Default value	Description
`glob`	String		Matches the path relative to the base directory using a `glob` pattern. See `FileSystem.getPathMatcher()` for the supported `glob` syntax. Required, unless `regex` is specified.
`regex`	`Pattern`		Matches the path relative to the directory using a regular expression. See `Pattern` for the supported regular expression syntax. Required, unless `glob` is specified.

Table 24. `IfFileName` nested elements
Type	Multiplicity	Description
`PathCondition`	zero or more	A set of optional nested conditions. This condition matches only if all the nested conditions also match.

📖 Plugin reference for IfFileName

`IfLastModified`

Accepts files based on their last modification timestamp.

Table 25. `IfLastModified` configuration attributes
Attribute	Type	Default value	Description
`age`	`Duration`		The condition accepts files that are as old or older than the specified duration. Required

Table 26. `IfLastModified` nested elements
Type	Multiplicity	Description
`PathCondition`	zero or more	A set of optional nested conditions. This condition matches only if all the nested conditions also match.

📖 Plugin reference for IfLastModified

`IfNot`

Negates the result of the nested condition.

Table 27. `IfNot` nested elements
Type	Multiplicity	Description
`PathCondition`	one	The path condition to negate.

📖 Plugin reference for IfNot

`IfAll`

Accepts a file if all the nested conditions are true.

Table 28. `IfAll` nested elements
Type	Multiplicity	Description
`PathCondition`	one or more	The nested conditions to check.

📖 Plugin reference for IfAll

`IfAny`

Table 29. `IfAny` nested elements
Type	Multiplicity	Description
`PathCondition`	one or more	The nested conditions to check.

📖 Plugin reference for IfAny

`ScriptCondition`

The ScriptCondition uses a JSR 223 script to determine a list of matching files.

Its configuration consists of a single nested script element:

Table 30. `ScriptCondition` nested elements
Type	Multiplicity	Description
`Script`, `ScriptFile` or `ScriptRef`	one	A reference to the script to execute. See Scripts for more details about scripting.

The script must return a list of PathWithAttributes objects and supports the following bindings:

Table 31. Script Bindings
Binding name	Type	Description
`basePath`	`Path`	The path of the base directory.
`configuration`	`Configuration`	The `Configuration` object.
`pathList`	`List<PathWithAttributes>`	The list of files contained in the base directory. The paths are obtained by resolving the relative file names against `basePath`.
`substitutor`	`StrSubstitutor`	The `StrSubstitutor` used to replace lookup variables.
`statusLogger`	`Logger`	The Status Logger to used by diagnostic messages in the script.
`<key>`	`String`	If `<key>` is not one of the above, it will be bound to the value given by the global `Properties` configuration element.

For an example of ScriptCondition usage, see the Using ScriptCondition example below.

📖 Plugin reference for ScriptCondition

Configuration recipes

`logrotate` equivalent configuration

Logrotate is a common UNIX utility that rotates log files.

Since a Java application cannot be notified that the log files need to be reloaded logrotate can be used with Java application through its copytruncate option (see logrotate(8) man page). A sample logrotate configuration file might therefore look like:

/var/log/app.log {
  copytruncate
  compress
  rotate 15
  daily
  maxsize 100k
}

This configuration has a problem, which is explained in the documentation of the copytruncate option:

Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost.

Fortunately you can replace the usage of logrotate with a rolling file appender. An equivalent configuration will look like this:

XML
JSON
YAML
Properties

Snippet from an example log4j2.xml

<RollingFile name="FILE"
             fileName="/var/log/app.log"
             filePattern="/var/log/app.log.%i.gz"> (1)
  <JsonTemplateLayout/>
  <DefaultRolloverStrategy max="15"/> (2)
  <Policies>
    <CronTriggeringPolicy schedule="0 0 0 * * ?"/> (3)
    <SizeBasedTriggeringPolicy size="100k"/> (4)
  </Policies>
</RollingFile>

Snippet from an example log4j2.json

"RollingFile": {
  "name": "FILE",
  "fileName": "/var/log/app.log",
  "filePattern": "/var/log/app.log.%i.gz", (1)
  "JsonTemplateLayout": {},
  "DefaultRolloverStrategy": {
    "max": 15 (2)
  },
  "Policies": {
    "CronTriggeringPolicy": {
      "schedule": "0 0 0 * * ?" (3)
    },
    "SizeBasedTriggeringPolicy": {
      "size": "100k" (4)
    }
  }
}

Snippet from an example log4j2.yaml

RollingFile:
  name: "FILE"
  fileName: "/var/log/app.log"
  filePattern: "/var/log/app.log.%i.gz" (1)
  JsonTemplateLayout: {}
  DefaultRolloverStrategy:
    max: 15 (2)
  Policies:
    CronTriggeringPolicy:
      schedule: "0 0 0 * * ?" (3)
    SizeBasedTriggeringPolicy:
      size: "100k" (4)

Snippet from an example log4j2.properties

appender.0.type = RollingFile
appender.0.name = FILE
appender.0.fileName = /var/log/app.log
(1)
appender.0.filePattern = /var/log/app.log.%i.gz

appender.0.layout.type = JsonTemplateLayout

appender.0.strategy.type = DefaultRolloveStrategy
(2)
appender.0.strategy.max = 15

appender.0.policy.type = Policies
appender.0.policy.0.type = CronTriggeringPolicy
(3)
appender.0.policy.0.schedule = 0 0 0 * * ?
appender.0.policy.1.type = SizeBasedTriggeringPolicy
(4)
appender.0.policy.1.size = 100k

1	Equivalent to `compress`: archived files are compressed.
2	Equivalent to `rotate 15`: only the `15` latest log files are kept.
3	Equivalent to `daily`: logs will be rotated at midnight of each day.
4	Equivalent to `maxsize 100k`: logs will be rotated if they exceed 100 kB of size.

Timestamped log file names

The following configuration creates one log file every day and deletes those more than 15 days old.

Since we have a %d pattern in the configuration file, we need to use an explicit Delete action.

XML
JSON
YAML
Properties

Snippet from an example log4j2.xml

<RollingFile name="FILE"
             filePattern="/var/log/app.%d{yyyy-MM-dd}.log.gz"> (1)
  <JsonTemplateLayout/>
  <DirectWriteRolloverStrategy>
    <Delete basePath="/var/log"> (2)
      <IfFileName regex="app\.\d{4}-\d{2}-\d{2}\.log\.gz"/> (3)
      <IfLastModified age="P15D"/>
    </Delete>
  </DirectWriteRolloverStrategy>
  <TimeBasedTriggeringPolicy/>
</RollingFile>

Snippet from an example log4j2.json

"RollingFile": {
  "name": "FILE",
  "filePattern": "/var/log/app.%d{yyyy-MM-dd}.log.gz", (1)
  "JsonTemplateLayout": {},
  "DirectWriteRolloverStrategy": {
    "Delete": { (2)
      "basePath": "/var/log",
      "IfFileName": {
        "regex": "app\\.\\d{4}-\\d{2}-\\d{2}\\.log\\.gz" (3)
      },
      "IfLastModified": {
        "age": "P15D"
      }
    }
  },
  "TimeBasedTriggeringPolicy": {}
}

Snippet from an example log4j2.yaml

RollingFile:
  name: "FILE"
  filePattern: "/var/log/app.%d{yyyy-MM-dd}.log.gz" (1)
  JsonTemplateLayout: {}
  DirectWriteRolloverStrategy:
    Delete: (2)
      basePath: "/var/log"
      IfFileName:
        regex: "app\\.\\d{4}-\\d{2}-\\d{2}\\.log\\.gz" (3)
      IfLastModified:
        age: "P15D"
  TimeBasedTriggeringPolicy: {}

Snippet from an example log4j2.properties

appender.0.type = RollingFile
appender.0.name = FILE
(1)
appender.0.filePattern = /var/log/app.%d{yyyy-MM-dd}.log.gz

appender.0.layout.type = JsonTemplateLayout

appender.0.strategy.type = DirectWriteRolloverStrategy
(2)
appender.0.strategy.delete.type = Delete
appender.0.strategy.delete.basePath = /var/log
appender.0.strategy.delete.0.type = IfFileName
(3)
appender.0.strategy.delete.0.regex = app\\.\\d{4}-\\d{2}-\\d{2}\\.log\\.gz
appender.0.strategy.delete.1.type = IfLastModified
appender.0.strategy.delete.1.age = P15D

appender.0.policy.type = TimeBaseTriggeringPolicy

1	Only the `filePattern` attribute is used, since we use the `DirectWriteRolloverStrategy`.
2	An explicit `Delete` action is provided.
3	You can select the files to delete using a regular expression or a simpler `app.*.log.gz` glob pattern.

Using `ScriptCondition`

If the supplied path conditions are not sufficient, you can use a ScriptCondition with an arbitrary script.

The example above can be rewritten into the following Groovy script:

Snippet from an example script-condition.groovy

def limit = FileTime.from(ZonedDateTime.now().minusDays(15).toInstant())
def matcher = FileSystems.getDefault().getPathMatcher('glob:app.*.log.gz')
statusLogger.info("Deleting files older than {}.", limit) (1)
return pathList.stream()
        .filter({
            def relPath = basePath.relativize(it.path) (2)
            def lastModified = it.attributes.lastModifiedTime()
            Files.isRegularFile(it.path)
                    && lastModified <= limit (3)
                    && matcher.matches(relPath) (4)
        })
        .collect(Collectors.toList())

1	Adding Status Logger calls in your script might help in debugging it.
2	`PathWithAttributes.getPath()` always starts with `basePath`, so we need to relativize it.
3	Equivalent to the `IfLastModified` condition.
4	Equivalent to the `IfFileName` condition.

You can use the script in your configuration file as follows:

XML
JSON
YAML
Properties

Snippet from an example log4j2.xml

<RollingFile name="FILE"
             filePattern="/var/log/app.%d{yyyy-MM-dd}.log.gz">
  <JsonTemplateLayout/>
  <DirectWriteRolloverStrategy>
    <Delete basePath="/var/log">
      <ScriptCondition>
        <ScriptFile path="script-condition.groovy"
                    language="groovy"/>
      </ScriptCondition>
    </Delete>
  </DirectWriteRolloverStrategy>
  <TimeBasedTriggeringPolicy/>
</RollingFile>

Snippet from an example log4j2.json

"RollingFile": {
  "name": "FILE",
  "filePattern": "/var/log/app.%d{yyyy-MM-dd}.log.gz",
  "JsonTemplateLayout": {},
  "DirectWriteRolloverStrategy": {
    "Delete": {
      "basePath": "/var/log",
      "ScriptCondition": {
        "ScriptFile": {
          "path": "script-condition.groovy",
          "language": "groovy"
        }
      }
    }
  },
  "TimeBasedTriggeringPolicy": {}
}

Snippet from an example log4j2.yaml

RollingFile:
  name: "FILE"
  filePattern: "/var/log/app.%d{yyyy-MM-dd}.log.gz"
  JsonTemplateLayout: {}
  DirectWriteRolloverStrategy:
    Delete:
      basePath: "/var/log"
      ScriptCondition:
        path: "script-condition.groovy"
        language: "groovy"
  TimeBasedTriggeringPolicy: {}

Snippet from an example log4j2.properties

appender.0.type = RollingFile
appender.0.name = FILE
appender.0.filePattern = /var/log/app.%d{yyyy-MM-dd}.log.gz

appender.0.layout.type = JsonTemplateLayout

appender.0.strategy.type = DirectWriteRolloverStrategy
appender.0.strategy.delete.type = Delete
appender.0.strategy.delete.basePath = /var/log
appender.0.strategy.delete.condition.type = ScriptCondition
appender.0.strategy.delete.condition.script.type = ScriptFile
appender.0.strategy.delete.condition.script.path = script-condition.groovy
appender.0.strategy.delete.condition.script.language = groovy

appender.0.policy.type = TimeBaseTriggeringPolicy

Separate folder per month

We can also create separate folders for temporarily related files. In the example below, we create a different folder for each month:

XML
JSON
YAML
Properties

Snippet from an example log4j2.xml

<RollingFile name="FILE"
             filePattern="/var/log/app/%d{yyyy-MM}/%d{yyyy-MM-dd}.log.gz"> (1)
  <JsonTemplateLayout/>
  <DirectWriteRolloverStrategy>
    <Delete basePath="/var/log/app"
            maxDepth="2"> (2)
      <IfLastModified age="P90D"/>
    </Delete>
  </DirectWriteRolloverStrategy>
  <TimeBasedTriggeringPolicy/>
</RollingFile>

Snippet from an example log4j2.json

"RollingFile": {
  "name": "FILE",
  "filePattern": "/var/log/app/%{yyyy-MM}/%d{yyyy-MM-dd}.log.gz", (1)
  "JsonTemplateLayout": {},
  "DirectWriteRolloverStrategy": {
    "Delete": {
      "basePath": "/var/log/app",
      "maxDepth": 2, (2)
      "IfLastModified": {
        "age": "P90D"
      }
    }
  },
  "TimeBasedTriggeringPolicy": {}
}

Snippet from an example log4j2.yaml

RollingFile:
  name: "FILE"
  filePattern: "/var/log/app/%d{yyyy-MM}/%d{yyyy-MM-dd}.log.gz" (1)
  JsonTemplateLayout: {}
  DirectWriteRolloverStrategy:
    Delete:
      basePath: "/var/log/app"
      maxDepth: 2 (2)
      IfLastModified:
        age: "P90D"
  TimeBasedTriggeringPolicy: {}

Snippet from an example log4j2.properties

appender.0.type = RollingFile
appender.0.name = FILE
(1)
appender.0.filePattern = /var/log/app/%d{yyyy-MM]/%d{yyyy-MM-dd}.log.gz

appender.0.layout.type = JsonTemplateLayout

appender.0.strategy.type = DirectWriteRolloverStrategy
appender.0.strategy.delete.type = Delete
appender.0.strategy.delete.basePath = /var/log/app
(2)
appender.0.strategy.delete.maxDepth = 2
appender.0.strategy.delete.0.type = IfLastModified
appender.0.strategy.delete.0.age = P90D

appender.0.policy.type = TimeBaseTriggeringPolicy

1	We use two `%d` patterns to specify the folder and file name.
2	We increase the recursion depth of the `Delete` action to extend to subfolders of the base directory.

Rolling file appenders

Appenders

Common configuration

Conversion patterns

RollingFile configuration

RollingRandomAccessFile configuration

Triggering Policies

Common concerns

OnStartupTriggeringPolicy

CronTriggeringPolicy

SizeBasedTriggeringPolicy

TimeBasedTriggeringPolicy

Multiple policies

Rollover strategies

Common configuration

DefaultRolloverStrategy configuration

DirectWriteRolloverStrategy configuration

Incrementing file indexes

Compressing archived files

Optional actions

Common action configuration

Delete action

PosixViewAttribute action

Path conditions

IfAccumulatedFileSize

IfAccumulatedFileCount

IfFileName

IfLastModified

IfNot

IfAll

IfAny

ScriptCondition

Configuration recipes

logrotate equivalent configuration

Timestamped log file names

Using ScriptCondition

Separate folder per month

`RollingFile` configuration

`RollingRandomAccessFile` configuration

`OnStartupTriggeringPolicy`

`CronTriggeringPolicy`

`SizeBasedTriggeringPolicy`

`TimeBasedTriggeringPolicy`

`DefaultRolloverStrategy` configuration

`DirectWriteRolloverStrategy` configuration

`Delete` action

`PosixViewAttribute` action

`IfAccumulatedFileSize`

`IfAccumulatedFileCount`

`IfFileName`

`IfLastModified`

`IfNot`

`IfAll`

`IfAny`

`ScriptCondition`

`logrotate` equivalent configuration

Using `ScriptCondition`