A classic batch is the processing of a file, for which records are read, for each record the data are processed and are persisted on database (reader, processor and writer).
In case the file is large and contains thousands of records, partial commits must be expected during processing. For example, every 1000 records, we can decide to commit the processing on the database.
Through Spring Batch it is very easy to get partial commits, it’s a simple parameter that is passed to the StepBuilder. In the case that it’s necessary to implement more complex partial commit policies, it is possible to implement custom completition policies (which in this case we will not see because it is not the subject of this article).
Finally, if a batch fails after partial commit, it must be possible to start again with the processing of the file by skipping the lines already committed. This last thing is also expected by Spring Batch, but we must add a few lines of code, it is not a simple configuration parameter. Also in this case it’s possible to implement custom skip and retry policies (and also in this case we will not see why not the subject of this article).
The environment is as follows:
- Java7
- Spring Boot 1.1.8
- Spring Batch 3.0
There are two concepts, the job instance and job execution. An instance of a job is accomplished through n executions (typically one, or more if there have been failures). Furthermore, only a failed job can be restarted.
To implement the restart we need the jobRegistry, jobOperator, jobExplorer and jobLauncher. Here is the complete code of the batch configuration.
Basically these are the steps:
- register the job in the jobRegistry
- get job instances through the jobOperator
- given the last instance, get executions through the jobOperator
- through the jobExplorer check if the last execution has failed
- in case the last execution has failed, the job must be restarted via the jobOperator
- in case the last execution was successful, launch a new job instance via the jobLauncher
Spring Batch takes care of managing the restart starting from the first uncommitted record.
Thanks Spring Batch 🙂
Giancarlo Parma 