Rolling file appenders

Log4j Core provides multiple appenders that allow to archive the current log file and truncate it.

The rolling file appenders support many configuration options.

If you are just interested in some configuration examples, see the Configuration recipes section.

Appenders

Log4j Core provides two rolling file appenders:

RollingFile

The RollingFile Appender uses FileOutputStream to access log files.

RollingRandomAccessFile

The RollingRandomAccessFile Appender uses RandomAccessFile to access log files.

Two appenders, even from different logger contexts, share a common RollingFileManager if:

Sharing a RollingFileManager guarantees that rollovers will occur only once, but requires most of the remaining configuration parameters to be the same.

Common configuration

Table 1. Common configuration attributes
Attribute Type Default value Description

Required

filePattern

Path

The pattern for the archived log files. See Conversion patterns for details

If fileName is null, the file pattern will also be used for the current file.

name

String

The name of the appender.

Optional

bufferSize

int

8192

The size of the ByteBuffer internally used by the appender.

See Buffering for more details.

bufferedIo

boolean

true

If set to true, Log4j Core will format each log event in an internal buffer, before sending it to the underlying resource.

The RandomAccessRollingFile Appender always enables the internal buffer.

See Buffering for more details.

createOnDemand

boolean

false

The appender creates the file on-demand. The appender only creates the file when a log event passes all filters and is routed to this appender. Defaults to false.

fileName

Path

null

The path to the current log file, can be null. If the folder containing the file does not exist, it will be created.

This attribute should not contain any runtime lookups nor pattern converters.

To use pattern converters, see filePattern.

filePermissions

PosixFilePermissions

null

If not null, it specifies the POSIX file permissions to apply to each created file. The permissions must be provided in the format used by PosixFilePermissions.fromString(), e.g. rw-rw----.

The underlying files system shall support POSIX file attribute view.

fileOwner

String

null

If not null, it specifies the file owner to apply to each created file.

The underlying files system shall support file owner attribute view.

fileGroup

String

null

If not null, it specifies the file group owner to apply to each created file.

The underlying files system shall support POSIX file attribute view.

ignoreExceptions

boolean

true

If false, logging exception will be forwarded to the caller of the logging statement. Otherwise, they will be ignored.

Logging exceptions are always also logged to Status Logger

immediateFlush

boolean

true

If set to true, the appender will flush its internal buffer after each event.

See Buffering for more details.

Table 2. Common nested elements
Type Multiplicity Description

Filter

zero or one

Allows filtering log events just before they are formatted and sent.

See also appender filtering stage.

Layout

zero or one

Formats log events.

See Layouts for more information.

TriggeringPolicy

one

Determines when to archive the current log file.

See Triggering Policies for more information.

RolloverStrategy

zero or one

Determines the actions performed during a rollover.

See Rollover strategies for more information.

Conversion patterns

The filePattern attribute accepts the following pattern specifiers:

All patterns accept also format specifiers, e.g. %03i prints the index as zero-padded number with 3 digits.

The $${date:...} runtime lookup and %d{...} conversion pattern are not equivalent:

  • $${date:...} formats the current date.

  • %d{...} formats the date of the previous rollover.

Setting the filePattern property to

$${date:yyyy-MM}/%d{yyyy-MM-dd}.log

will write the 2024-06-30.log log file into the 2024-07 directory. Consider using

%d{yyyy-MM}/%d{yyyy-MM-dd}.log

instead.

RollingFile configuration

The RollingFile Appender provides the following configuration options, beyond the common ones:

Table 3. RollingFile configuration attributes
Attribute Type Default value Description

append

boolean

true

If true, the log file will be opened in APPEND mode.

On most systems this guarantees atomic writes to the end of the file, even if the file is opened by multiple applications.

locking

boolean

false

If true, Log4j will lock the log file at each log event.

Note that the effects of this setting depend on the Operating System: some systems like most POSIX OSes do not offer mandatory locking, but only advisory file locking.

This setting can also reduce the performance of the appender.

RollingRandomAccessFile configuration

The RollingRandomAccessFile Appender provides the following configuration options, beyond the common ones:

Table 4. RollingRandomAccessFile configuration attributes
Attribute Type Default value Description

append

boolean

true

If true, the appender starts writing at the end of the file.

This setting does not give the same atomicity guarantees as for the RollingFile Appender. The log file cannot be opened by multiple applications at the same time.

Triggering Policies

Triggering policies are Log4j plugins that implement the TriggeringPolicy interface and are used to decide when is it time to rollover the current log file.

Common concerns

Triggering policies usually decide to rollover a file based on two parameters:

  • the size of the current file.

    While all JVMs can reliably check the size of a file, Log4j only checks it at startup and at each rollover and infers all file size changes, based on the size of logs delivered. If multiple managers are writing to the same file, the size computation will be off.

  • the creation timestamp of the current file.

    Not all file systems support a creation timestamp. Most notably, POSIX does not specify such a timestamp, so the value used by Log4j might depend on the type of filesystem used and the JVM. If the creation timestamp is not available, the last modification timestamp is used, which might differ by up to a whole rollover period.

    See BasicFileAttributes.creationTime() for more details.

    After the first rollover Log4j uses the timestamp of the rollover as creation timestamp.

OnStartupTriggeringPolicy

The OnStartupTriggeringPolicy policy causes a rollover only once during the lifetime of a JVM. A rollover will occur if the log file was created before the start time of the current JVM and the minimum file size is met or exceeded.

Table 5. OnStartupTriggeringPolicy configuration attributes
Attribute Type Default value Description

minSize

long

1

The minimum size the file must have to roll over.

The OnStartupTriggeringPolicy uses JMX to retrieve the start time of the JVM.

On platforms where JMX is absent or disabled, like Android or Google App Engine, the OnStartupTriggeringPolicy will use the timestamp, when its class was loaded instead.

CronTriggeringPolicy

The CronTriggeringPolicy triggers rollover based on a CRON expression.

The filePattern attribute of the Appender should contain a %d{...} timestamp, otherwise the target file will be overwritten on each rollover.

This policy is controlled by a timer and is asynchronous in processing log events, so it is possible that log events from the previous or next period may appear at the beginning or end of the log file.

Table 6. CronTriggeringPolicy configuration attributes
Attribute Type Default value Description

evaluateOnStartup

boolean

false

On startup the cron expression will be evaluated against the current file’s creation timestamp. If a rollover should have occurred between that time and the current time the file will be immediately rolled over.

schedule

CronExpression

0 0 0 * * ?

The cron expression, using the same syntax as the Quartz scheduler.

The supported fields are in order:

  • second

  • minute

  • hour

  • day of month

  • month

  • day of week

See CronExpression for a full specification of the format.

SizeBasedTriggeringPolicy

The SizeBasedTriggeringPolicy causes a rollover once the file has reached the specified size.

The size can be specified in bytes, with the suffix KB, MB, GB, or TB for example 20MB. size. The size may also contain a fractional value such as 1.5 MB. The size is evaluated using the Java root Locale so a period must always be used for the fractional unit.

When combined with a time-based triggering policy, the filePattern attribute of the Appender should contain an %i conversion pattern. Otherwise, the target file will be overwritten on each rollover.

Table 7. SizeBasedTriggeringPolicy configuration attributes
Attribute Type Default value Description

size

FileSize

10 MB

The maximum file size of the current log file. Once this size is reached, a rollover will occur.

The syntax of this attribute is:

<number> [ " " ] [ <unit> ]

i.e.:

  • a number recognized by NumberFormat

  • followed by optional whitespace

  • followed by an optional unit: k, kB, M, MB, G, GB, T, TB.

TimeBasedTriggeringPolicy

The TimeBasedTriggeringPolicy causes a rollover, when the smallest time unit in filePattern changes value. So the rollover can occur at the end of a month, week, at midnight, end of the hour, etc.

The filePattern attribute of the Appender should contain a %d{...} timestamp, otherwise the target file will be overwritten on each rollover.

Multiple %d patterns can be used. The rollover frequency time unit will be determined by the last %d pattern.

Table 8. TimeBasedTriggeringPolicy configuration attributes
Attribute Type Default value Description

interval

int

1

How often a rollover should occur based on the most specific time unit in the last %d{...} pattern.

modulate

boolean

false

If true, the rollover will be aligned to a boundary of interval time units.

For example, if the rotation frequency is hourly with an interval of 4 and the application starts at 3:14:

false

Will roll over at 3:00, 7:00, 11:00, 15:00, 19:00 and 23:00 each day.

true

Will roll over at 0:00, 4:00, 8:00, 12:00, 16:00 and 20:00 each day.

Only applies if interval is greater than 1.

maxRandomDelay

int

0

Indicates the maximum number of seconds to randomly delay a rollover.

This setting is useful on servers where multiple applications are configured to roll over log files at the same time and can spread the load of doing so across time.

When the rollover frequency is daily or lower, the rolling file appender will rotate files at midnight of the server’s default timezone.

You can modify the time of the rollover by specifying a different timezone in the %d pattern. See date converter syntax for more details.

Multiple policies

A rolling file appender allows only one nested triggering policy element. If you wish to use multiple policies, you need to wrap them in a Policies element. The element itself has no configuration attributes.

The effects of using both time-based triggering policies (CronTriggeringPolicy and TimeBasedTriggeringPolicy) at the same time are undefined.

For example, the following XML fragment defines policies that rollover the log:

  • when the JVM starts.

  • when the log size reaches 10 MB.

  • at midnight local time.

  • XML

  • JSON

  • YAML

  • Properties

Snippet from an example log4j2.xml
<RollingFile name="FILE"
             fileName="app.log"
             filePattern="app.%d{yyyy-MM-dd}.%i.log">
  <JsonTemplateLayout/>
  <Policies>
    <OnStartupTriggeringPolicy/>
    <SizeBasedTriggeringPolicy/>
    <TimeBasedTriggeringPolicy/>
  </Policies>
</RollingFile>
Snippet from an example log4j2.json
"RollingFile": {
  "name": "FILE",
  "fileName": "app.log",
  "filePattern": "app.%d{yyyy-MM-dd}.%i.log",
  "JsonTemplateLayout": {},
  "Policies": {
    "OnStartupTriggeringPolicy": {},
    "SizeBasedTriggeringPolicy": {},
    "TimeBasedTriggeringPolicy": {}
  }
}
Snippet from an example log4j2.yaml
RollingFile:
  name: "FILE"
  fileName: "app.log"
  filePattern: "app.%d{yyyy-MM-dd}.%i.log"
  JsonTemplateLayout: {}
  Policies:
    OnStartupTriggeringPolicy: {}
    SizeBasedTriggeringPolicy: {}
    TimeBasedTriggeringPolicy: {}
Snippet from an example log4j2.properties
appender.0.type = RollingFile
appender.0.name = FILE
appender.0.fileName = app.log
appender.0.filePattern = app.%d{yyyy-MM-dd}.%i.log

appender.0.layout.type = JsonTemplateLayout

appender.0.policy.type = Policies
appender.0.policy.0.type = OnStartupTriggeringPolicy
appender.0.policy.1.type = SizeBasedTriggeringPolicy
appender.0.policy.2.type = TimeBasedTriggeringPolicy

Rollover strategies

A rollover strategy determines how old archive files are rotated or deleted to make place for newer ones. The actions performed during a rollover include:

There are two different rollover strategies available out of the box:

DefaultRolloverStrategy

This is the default strategy used if the fileName configuration attribute is specified. It stores the current log file in the location specified by fileName and the archived log files in the location specified by filePattern.

DirectWriteRolloverStrategy

This is the default strategy used if the fileName configuration attribute is null. It stores the current log file directly in its rolled over location specified by filePattern.

Using runtime lookups in the fileName configuration attribute is discouraged, since it will break the logic of the DefaultRolloverStrategy. Use the DirectWriteRolloverStrategy instead by omitting the fileName attribute in your configuration.

Common configuration

Both rollover strategies support the following configuration parameters:

Table 9. Common rollover strategy configuration attributes
Attribute Type Default value Description

compressionLevel

int

7

Determine the compression level of archived log files.

See Compressing archived files for more details.

tempCompressedFilePattern

Path

Temporary location to store compressed files.

See Compressing archived files for more details.

stopCustomActionsOnError

boolean

true

If true, custom actions will be stopped if one of them fails.

Table 10. Common Rollover Strategy nested elements
Type Multiplicity Description

Action

zero or more

Additional actions to perform on a rollover beyond file rotation and compression.

See Optional actions for more details.

DefaultRolloverStrategy configuration

The DefaultRolloverStrategy supports additional attributes that control the incrementation of file indexes.

Table 11. DefaultRolloverStrategy configuration attributes
Attribute Type Default value Description

fileIndex

enumeration

max

Determines the strategy for determining the value of the %i conversion pattern for the current log file.

The supported values are:

min

uses the value of the min attribute.

max

uses the first unused integer between the min and max attributes.

nomax

uses the first unused integer not lower than the value of the min attribute.

min

int

1

Minimum value for the %i conversion pattern.

max

int

7

Maximum value for the %i conversion pattern.

This attribute is ignored if fileIndex is set to nomax.

DirectWriteRolloverStrategy configuration

The DirectWriteRolloverStrategy has a reduced set of configuration options for incrementing the file index:

  • the minimum file index is always 1.

  • the incrementing strategy is always max.

  • the maximum file index can be configured using the following configuration attribute:

    Table 12. DirectWriteRolloverStrategy configuration attributes
    Attribute Type Default value Description

    maxFiles

    int

    7

    Maximum value for the %i conversion pattern.

Incrementing file indexes

During a rollover event, the current log file is moved to the location determined by evaluating filePattern, using min as value for the %i conversion pattern. If an old log file is already present in that location, the rolling file appender:

  1. Tries to increment the value of %i using the strategy specified by the fileIndex configuration attribute.

  2. If that fails, it removes the oldest log file and rotates the remaining ones to make place for a new log archive.

There are three strategies available:

min

Using the min strategy, the newest log file will have index min and the oldest one will have index max.

This is the log rotation strategy used by traditional UNIX tools and by Log4j 1. It is not the default strategy of Log4j 2.

Assuming min="1" and max="3" the rotation of the log files is represented in the graph below:

@startuml
class "Initial status" as initial {
  <color:green>app.log</color>
}
class "1st rollover" as first {
  <color:green>app.log</color>
  app.1.log
}
class "2nd rollover" as second {
  <color:green>app.log</color>
  app.1.log
  app.2.log
}
class "3rd rollover" as third {
  <color:green>app.log</color>
  app.1.log
  app.2.log
  app.3.log
}
class "4th rollover" as fourth {
  <color:green>app.log</color>
  app.1.log
  app.2.log
  app.3.log
}

object "Delete file" as delete

initial::app.log -> first::app.1.log

first::app.log -> second::app.1.log
first::app.1.log -> second::app.2.log

second::app.log -> third::app.1.log
second::app.1.log -> third::app.2.log
second::app.2.log -> third::app.3.log

third::app.log -> fourth::app.1.log
third::app.1.log -> fourth::app.2.log
third::app.2.log -> fourth::app.3.log
third::app.3.log -[#red]-> delete
@enduml
max

Using the max strategy, the oldest log file will have index min and the newest one will have index max.

This is the default strategy since Log4j 2.

Assuming min="1" and max="3" the rotation of the log files is represented in the graph below:

@startuml
class "Initial status" as initial {
  <color:green>app.log</color>
}
class "1st rollover" as first {
  <color:green>app.log</color>
  app.1.log
}
class "2nd rollover" as second {
  <color:green>app.log</color>
  app.1.log
  app.2.log
}
class "3rd rollover" as third {
  <color:green>app.log</color>
  app.1.log
  app.2.log
  app.3.log
}
class "4th rollover" as fourth {
  <color:green>app.log</color>
  app.1.log
  app.2.log
  app.3.log
}

object "Delete file" as delete

initial::app.log -> first::app.1.log

first::app.log -> second::app.2.log
first::app.1.log -> second::app.1.log

second::app.log -> third::app.3.log
second::app.1.log -> third::app.1.log
second::app.2.log -> third::app.2.log

third::app.log -> fourth::app.3.log
third::app.1.log -[#red]-> delete
third::app.2.log -> fourth::app.1.log
third::app.3.log -> fourth::app.2.log
@enduml
nomax

Using the nomax strategy no files will ever be deleted and newer archive files will be assigned increasing index numbers, starting from min.

@startuml
class "Initial status" as initial {
  <color:green>app.log</color>
}
class "1st rollover" as first {
  <color:green>app.log</color>
  app.1.log
}
class "2nd rollover" as second {
  <color:green>app.log</color>
  app.1.log
  app.2.log
}
class "3rd rollover" as third {
  <color:green>app.log</color>
  app.1.log
  app.2.log
  app.3.log
}
class "4th rollover" as fourth {
  <color:green>app.log</color>
  app.1.log
  app.2.log
  app.3.log
  app.4.log
}

initial::app.log -> first::app.1.log

first::app.log -> second::app.2.log
first::app.1.log -> second::app.1.log

second::app.log -> third::app.3.log
second::app.1.log -> third::app.1.log
second::app.2.log -> third::app.2.log

third::app.log -> fourth::app.4.log
third::app.1.log -> fourth::app.1.log
third::app.2.log -> fourth::app.2.log
third::app.3.log -> fourth::app.3.log
@enduml

Compressing archived files

When the current log file is archived, the rolling file appender can compress it. Compression is activated, based on the extension of the archived file name. The following extensions are recognized:

Extension Supports compressionLevel Description

.zip

ZIP archive using the DEFLATE algorithm

.gz

GZIP archive using the DEFLATE algorithm

.bz2dep

BZip2 algorithm

.deflatedep

DEFLATE algorithm

.pack200dep

Pack200 algorithm

.xz dep

XZ algorithm

.zst dep

ZStandard algorithm

If the tempCompressedFilePattern attribute is set, the current log file:

  • will be compressed and stored in the location given by tempCompressedFilePattern

  • and then it will be moved to the location given by filePattern.

dep

Additional dependencies are required to use these compression algorithms:

  • Maven

  • Gradle

<dependency>
  <groupId>org.apache.commons</groupId>
  <artifactId>commons-compress</artifactId>
  <version>1.27.1</version>
  <scope>runtime</scope>
</dependency>
runtimeOnly 'org.apache.commons:commons-compress:1.27.1'

The .xz and .zst extensions require additional dependencies. See Commons Compress documentation for more details.

Optional actions

The rotation of files and compression is always handled automatically by the rolling file appenders. Since Log4j 2.6, additional actions can be configured manually.

Log4j Core provides out-of-the-box two actions:

Common action configuration

Both actions support the following configuration attributes:

Table 13. Common action configuration attributes
Attribute Type Default value Description

basePath

Path

It sets the base directory for the action. The action will be limited to files in this directory, and all paths will be relative to this directory.

Required

followLinks

boolean

false

If set to true, the action will follow symbolic links.

Setting this value to true will allow the action to access files outside basePath, which might introduce a security risk.

maxDepth

int

1

The maximum number of directory levels to visit. By default, the action does not recurse into subdirectories of basePath.

Table 14. Common action nested elements
Type Multiplicity Description

PathCondition

zero or more

A set of path conditions. The action will only be performed on files for which the condition returns true.

Delete action

The Delete Action deletes old log files that match the configured condition.

A limited version of this action is performed automatically to increment file indexes.

You only need an explicit Delete Action if you use a %d pattern.

Besides the common configuration options the Delete Action also supports the following options:

Table 15. Delete Action configuration attributes
Attribute Type Default value Description

testMode

boolean

false

If true, no files will be deleted, but a Status Logger message of level INFO will be issued instead.

Table 16. Delete Action nested elements
Type Multiplicity Description

PathCondition

zero or more

If present, the action will only be performed on files for which the condition returns true.

Required, unless a ScriptCondition is provided, in which case these elements are ignored.

PathSorter

zero or one

Provides a sorting order for files contained in basePath.

The default implementation is PathSortByModificationTime, which sorts files by ascending modification timestamp.

ScriptCondition

zero or one

If present, all the nested PathConditions are ignored and the provided script is executed to select the file to delete.

PosixViewAttribute action

This action allows modifying the POSIX attributes (owner, group and permissions) of archive log files.

By default, the POSIX attributes are inherited from the current log file.

This action is only necessary if archived log files must have different attributes.

Besides the common configuration options the PosixViewAttribute Action also supports the following options.

Table 17. PosixViewAttribute Action configuration attributes
Attribute Type Default value Description

filePermissions

PosixFilePermissions

null

If not null, it specifies the POSIX file permissions to apply to each created file. The permissions must be provided in the format used by PosixFilePermissions.fromString(), e.g. rw-rw----.

The underlying files system shall support POSIX file attribute view.

fileOwner

String

null

If not null, it specifies the file owner to apply to each created file.

The underlying files system shall support file owner attribute view.

fileGroup

String

null

If not null, it specifies the file group owner to apply to each created file.

The underlying files system shall support POSIX file attribute view.

Table 18. PosixViewAttribute Action nested elements
Type Multiplicity Description

PathCondition

one or more

A set of conditions to select the files to modify.

Path conditions

To select the files for additional actions, Log4j provides the following path conditions:

IfAccumulatedFileSize

When evaluated on a list of files, this condition sums the size of each file with the size of the preceding files. It returns true for files that exceed a configurable threshold.

The result of this condition depends on the sorting order on files. See PathSorter for more information.

Table 19. IfAccumulatedFileSize configuration attributes
Attribute Type Default value Description

exceeds

FileSize

The threshold size for the condition to match.

Admits the same syntax as the size attribute of SizeBasedTriggeringPolicy.

Required

Table 20. IfAccumulatedFileSize nested elements
Type Multiplicity Description

PathCondition

zero or more

A set of optional nested conditions. This condition matches only if all the nested conditions also match.

IfAccumulatedFileCount

When evaluated on a list of files, this condition returns true if the 1-based index of a file exceeds a configurable threshold.

The result of this condition depends on the sorting order on files. See PathSorter for more information.

Table 21. IfAccumulatedFileCount configuration attributes
Attribute Type Default value Description

exceeds

int

The threshold for the condition to match.

Required

Table 22. IfAccumulatedFileCount nested elements
Type Multiplicity Description

PathCondition

zero or more

A set of optional nested conditions. This condition matches only if all the nested conditions also match.

IfFileName

Matches files based on their path relative to the base directory.

Table 23. IfFileName configuration attributes
Attribute Type Default value Description

glob

String

Matches the path relative to the base directory using a glob pattern.

See FileSystem.getPathMatcher() for the supported glob syntax.

Required, unless regex is specified.

regex

Pattern

Matches the path relative to the directory using a regular expression.

See Pattern for the supported regular expression syntax.

Required, unless glob is specified.

Table 24. IfFileName nested elements
Type Multiplicity Description

PathCondition

zero or more

A set of optional nested conditions. This condition matches only if all the nested conditions also match.

IfLastModified

Accepts files based on their last modification timestamp.

Table 25. IfLastModified configuration attributes
Attribute Type Default value Description

age

Duration

The condition accepts files that are as old or older than the specified duration.

Required

Table 26. IfLastModified nested elements
Type Multiplicity Description

PathCondition

zero or more

A set of optional nested conditions. This condition matches only if all the nested conditions also match.

IfNot

Negates the result of the nested condition.

Table 27. IfNot nested elements
Type Multiplicity Description

PathCondition

one

The path condition to negate.

IfAll

Accepts a file if all the nested conditions are true.

Table 28. IfAll nested elements
Type Multiplicity Description

PathCondition

one or more

The nested conditions to check.

IfAny

Table 29. IfAny nested elements
Type Multiplicity Description

PathCondition

one or more

The nested conditions to check.

ScriptCondition

The ScriptCondition uses a JSR 223 script to determine a list of matching files.

Its configuration consists of a single nested script element:

Table 30. ScriptCondition nested elements
Type Multiplicity Description

Script, ScriptFile or ScriptRef

one

A reference to the script to execute.

See Scripts for more details about scripting.

The script must return a list of PathWithAttributes objects and supports the following bindings:

Table 31. Script Bindings
Binding name Type Description

basePath

Path

The path of the base directory.

configuration

Configuration

The Configuration object.

pathList

List<PathWithAttributes>

The list of files contained in the base directory.

The paths are obtained by resolving the relative file names against basePath.

substitutor

StrSubstitutor

The StrSubstitutor used to replace lookup variables.

statusLogger

Logger

The Status Logger to used by diagnostic messages in the script.

<key>

String

If <key> is not one of the above, it will be bound to the value given by the global Properties configuration element.

For an example of ScriptCondition usage, see the Using ScriptCondition example below.

Configuration recipes

logrotate equivalent configuration

Logrotate is a common UNIX utility that rotates log files.

Since a Java application cannot be notified that the log files need to be reloaded logrotate can be used with Java application through its copytruncate option (see logrotate(8) man page). A sample logrotate configuration file might therefore look like:

/var/log/app.log {
  copytruncate
  compress
  rotate 15
  daily
  maxsize 100k
}

This configuration has a problem, which is explained in the documentation of the copytruncate option:

Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost.

Fortunately you can replace the usage of logrotate with a rolling file appender. An equivalent configuration will look like this:

  • XML

  • JSON

  • YAML

  • Properties

Snippet from an example log4j2.xml
<RollingFile name="FILE"
             fileName="/var/log/app.log"
             filePattern="/var/log/app.log.%i.gz"> (1)
  <JsonTemplateLayout/>
  <DefaultRolloverStrategy max="15"/> (2)
  <Policies>
    <CronTriggeringPolicy schedule="0 0 0 * * ?"/> (3)
    <SizeBasedTriggeringPolicy size="100k"/> (4)
  </Policies>
</RollingFile>
Snippet from an example log4j2.json
"RollingFile": {
  "name": "FILE",
  "fileName": "/var/log/app.log",
  "filePattern": "/var/log/app.log.%i.gz", (1)
  "JsonTemplateLayout": {},
  "DefaultRolloverStrategy": {
    "max": 15 (2)
  },
  "Policies": {
    "CronTriggeringPolicy": {
      "schedule": "0 0 0 * * ?" (3)
    },
    "SizeBasedTriggeringPolicy": {
      "size": "100k" (4)
    }
  }
}
Snippet from an example log4j2.yaml
RollingFile:
  name: "FILE"
  fileName: "/var/log/app.log"
  filePattern: "/var/log/app.log.%i.gz" (1)
  JsonTemplateLayout: {}
  DefaultRolloverStrategy:
    max: 15 (2)
  Policies:
    CronTriggeringPolicy:
      schedule: "0 0 0 * * ?" (3)
    SizeBasedTriggeringPolicy:
      size: "100k" (4)
Snippet from an example log4j2.properties
appender.0.type = RollingFile
appender.0.name = FILE
appender.0.fileName = /var/log/app.log
(1)
appender.0.filePattern = /var/log/app.log.%i.gz

appender.0.layout.type = JsonTemplateLayout

appender.0.strategy.type = DefaultRolloveStrategy
(2)
appender.0.strategy.max = 15

appender.0.policy.type = Policies
appender.0.policy.0.type = CronTriggeringPolicy
(3)
appender.0.policy.0.schedule = 0 0 0 * * ?
appender.0.policy.1.type = SizeBasedTriggeringPolicy
(4)
appender.0.policy.1.size = 100k
1 Equivalent to compress: archived files are compressed.
2 Equivalent to rotate 15: only the 15 latest log files are kept.
3 Equivalent to daily: logs will be rotated at midnight of each day.
4 Equivalent to maxsize 100k: logs will be rotated if they exceed 100 kB of size.

Timestamped log file names

The following configuration creates one log file every day and deletes those more than 15 days old.

Since we have a %d pattern in the configuration file, we need to use an explicit Delete action.

  • XML

  • JSON

  • YAML

  • Properties

Snippet from an example log4j2.xml
<RollingFile name="FILE"
             filePattern="/var/log/app.%d{yyyy-MM-dd}.log.gz"> (1)
  <JsonTemplateLayout/>
  <DirectWriteRolloverStrategy>
    <Delete basePath="/var/log"> (2)
      <IfFileName regex="app\.\d{4}-\d{2}-\d{2}\.log\.gz"/> (3)
      <IfLastModified age="P15D"/>
    </Delete>
  </DirectWriteRolloverStrategy>
  <TimeBasedTriggeringPolicy/>
</RollingFile>
Snippet from an example log4j2.json
"RollingFile": {
  "name": "FILE",
  "filePattern": "/var/log/app.%d{yyyy-MM-dd}.log.gz", (1)
  "JsonTemplateLayout": {},
  "DirectWriteRolloverStrategy": {
    "Delete": { (2)
      "basePath": "/var/log",
      "IfFileName": {
        "regex": "app\\.\\d{4}-\\d{2}-\\d{2}\\.log\\.gz" (3)
      },
      "IfLastModified": {
        "age": "P15D"
      }
    }
  },
  "TimeBasedTriggeringPolicy": {}
}
Snippet from an example log4j2.yaml
RollingFile:
  name: "FILE"
  filePattern: "/var/log/app.%d{yyyy-MM-dd}.log.gz" (1)
  JsonTemplateLayout: {}
  DirectWriteRolloverStrategy:
    Delete: (2)
      basePath: "/var/log"
      IfFileName:
        regex: "app\\.\\d{4}-\\d{2}-\\d{2}\\.log\\.gz" (3)
      IfLastModified:
        age: "P15D"
  TimeBasedTriggeringPolicy: {}
Snippet from an example log4j2.properties
appender.0.type = RollingFile
appender.0.name = FILE
(1)
appender.0.filePattern = /var/log/app.%d{yyyy-MM-dd}.log.gz

appender.0.layout.type = JsonTemplateLayout

appender.0.strategy.type = DirectWriteRolloverStrategy
(2)
appender.0.strategy.delete.type = Delete
appender.0.strategy.delete.basePath = /var/log
appender.0.strategy.delete.0.type = IfFileName
(3)
appender.0.strategy.delete.0.regex = app\\.\\d{4}-\\d{2}-\\d{2}\\.log\\.gz
appender.0.strategy.delete.1.type = IfLastModified
appender.0.strategy.delete.1.age = P15D

appender.0.policy.type = TimeBaseTriggeringPolicy
1 Only the filePattern attribute is used, since we use the DirectWriteRolloverStrategy.
2 An explicit Delete action is provided.
3 You can select the files to delete using a regular expression or a simpler app.*.log.gz glob pattern.

Using ScriptCondition

If the supplied path conditions are not sufficient, you can use a ScriptCondition with an arbitrary script.

The example above can be rewritten into the following Groovy script:

Snippet from an example script-condition.groovy
def limit = FileTime.from(ZonedDateTime.now().minusDays(15).toInstant())
def matcher = FileSystems.getDefault().getPathMatcher('glob:app.*.log.gz')
statusLogger.info("Deleting files older than {}.", limit) (1)
return pathList.stream()
        .filter({
            def relPath = basePath.relativize(it.path) (2)
            def lastModified = it.attributes.lastModifiedTime()
            Files.isRegularFile(it.path)
                    && lastModified <= limit (3)
                    && matcher.matches(relPath) (4)
        })
        .collect(Collectors.toList())
1 Adding Status Logger calls in your script might help in debugging it.
2 PathWithAttributes.getPath() always starts with basePath, so we need to relativize it.
3 Equivalent to the IfLastModified condition.
4 Equivalent to the IfFileName condition.

You can use the script in your configuration file as follows:

  • XML

  • JSON

  • YAML

  • Properties

Snippet from an example log4j2.xml
<RollingFile name="FILE"
             filePattern="/var/log/app.%d{yyyy-MM-dd}.log.gz">
  <JsonTemplateLayout/>
  <DirectWriteRolloverStrategy>
    <Delete basePath="/var/log">
      <ScriptCondition>
        <ScriptFile path="script-condition.groovy"
                    language="groovy"/>
      </ScriptCondition>
    </Delete>
  </DirectWriteRolloverStrategy>
  <TimeBasedTriggeringPolicy/>
</RollingFile>
Snippet from an example log4j2.json
"RollingFile": {
  "name": "FILE",
  "filePattern": "/var/log/app.%d{yyyy-MM-dd}.log.gz",
  "JsonTemplateLayout": {},
  "DirectWriteRolloverStrategy": {
    "Delete": {
      "basePath": "/var/log",
      "ScriptCondition": {
        "ScriptFile": {
          "path": "script-condition.groovy",
          "language": "groovy"
        }
      }
    }
  },
  "TimeBasedTriggeringPolicy": {}
}
Snippet from an example log4j2.yaml
RollingFile:
  name: "FILE"
  filePattern: "/var/log/app.%d{yyyy-MM-dd}.log.gz"
  JsonTemplateLayout: {}
  DirectWriteRolloverStrategy:
    Delete:
      basePath: "/var/log"
      ScriptCondition:
        path: "script-condition.groovy"
        language: "groovy"
  TimeBasedTriggeringPolicy: {}
Snippet from an example log4j2.properties
appender.0.type = RollingFile
appender.0.name = FILE
appender.0.filePattern = /var/log/app.%d{yyyy-MM-dd}.log.gz

appender.0.layout.type = JsonTemplateLayout

appender.0.strategy.type = DirectWriteRolloverStrategy
appender.0.strategy.delete.type = Delete
appender.0.strategy.delete.basePath = /var/log
appender.0.strategy.delete.condition.type = ScriptCondition
appender.0.strategy.delete.condition.script.type = ScriptFile
appender.0.strategy.delete.condition.script.path = script-condition.groovy
appender.0.strategy.delete.condition.script.language = groovy

appender.0.policy.type = TimeBaseTriggeringPolicy

Separate folder per month

We can also create separate folders for temporarily related files. In the example below, we create a different folder for each month:

  • XML

  • JSON

  • YAML

  • Properties

Snippet from an example log4j2.xml
<RollingFile name="FILE"
             filePattern="/var/log/app/%d{yyyy-MM}/%d{yyyy-MM-dd}.log.gz"> (1)
  <JsonTemplateLayout/>
  <DirectWriteRolloverStrategy>
    <Delete basePath="/var/log/app"
            maxDepth="2"> (2)
      <IfLastModified age="P90D"/>
    </Delete>
  </DirectWriteRolloverStrategy>
  <TimeBasedTriggeringPolicy/>
</RollingFile>
Snippet from an example log4j2.json
"RollingFile": {
  "name": "FILE",
  "filePattern": "/var/log/app/%{yyyy-MM}/%d{yyyy-MM-dd}.log.gz", (1)
  "JsonTemplateLayout": {},
  "DirectWriteRolloverStrategy": {
    "Delete": {
      "basePath": "/var/log/app",
      "maxDepth": 2, (2)
      "IfLastModified": {
        "age": "P90D"
      }
    }
  },
  "TimeBasedTriggeringPolicy": {}
}
Snippet from an example log4j2.yaml
RollingFile:
  name: "FILE"
  filePattern: "/var/log/app/%d{yyyy-MM}/%d{yyyy-MM-dd}.log.gz" (1)
  JsonTemplateLayout: {}
  DirectWriteRolloverStrategy:
    Delete:
      basePath: "/var/log/app"
      maxDepth: 2 (2)
      IfLastModified:
        age: "P90D"
  TimeBasedTriggeringPolicy: {}
Snippet from an example log4j2.properties
appender.0.type = RollingFile
appender.0.name = FILE
(1)
appender.0.filePattern = /var/log/app/%d{yyyy-MM]/%d{yyyy-MM-dd}.log.gz

appender.0.layout.type = JsonTemplateLayout

appender.0.strategy.type = DirectWriteRolloverStrategy
appender.0.strategy.delete.type = Delete
appender.0.strategy.delete.basePath = /var/log/app
(2)
appender.0.strategy.delete.maxDepth = 2
appender.0.strategy.delete.0.type = IfLastModified
appender.0.strategy.delete.0.age = P90D

appender.0.policy.type = TimeBaseTriggeringPolicy
1 We use two %d patterns to specify the folder and file name.
2 We increase the recursion depth of the Delete action to extend to subfolders of the base directory.