Spring Boot, Spring Batch and exit codes

When creating batches to be invoked by a scheduler, it is very important to correctly manage the JVM exit codes.

Advertisements

When creating batches to be invoked by a scheduler, it is very important to correctly manage the JVM exit codes.
By convention, the JVM ends with an exit code equal to zero if there were no problems, otherwise with an exit code greater than zero.
In this way if the batch is not terminated correctly, interpreting the exit code, the scheduler can for example inform the application manager via email, or adopt strategies to relaunch or recover the batch itself, or terminate a job box.

If you use Spring Boot to start a Spring Batch-based batch, the JVM always ends with an exit code of zero, even in the case of runtime exceptions. In order to correctly manage the JVM exit codes, it is necessary to intervene by means of an ExitCodeGenerator.

The application stack is composed of:

Spring Core 4.0.7
Spring Boot 1.1.8
Spring Batch 3.0.1

in the class that configures the batch, we need to add the following methods:

@Bean public JobExecutionExitCodeGenerator jobExecutionExitCodeGenerator() {

return new JobExecutionExitCodeGenerator();

}

protected JobExecution addToJobExecutionExitCodeGenerator(JobExecution jobExecution) {

JobExecutionExitCodeGenerator jobExecutionExitCodeGenerator = jobExecutionExitCodeGenerator(); jobExecutionExitCodeGenerator.onApplicationEvent(new JobExecutionEvent(jobExecution)); return jobExecution;

}

as ExitCodeGenerator we can use the default implementation of Spring Boot which is JobExecutionExitCodeGenerator. So in the addToJobExecutionExitCodeGenerator method we pass the jobExecution to the exit code generator forcing the creation of the JobExecutionEvent event. When we launch the job, we must force the call to the addToJobExecutionExitCodeGenerator method:

addToJobExecutionExitCodeGenerator(jobLauncher.run(job(), jobParameters(jobParametersMap)));

In this way, when we end the batch in the Application class, the exit code will be the one actually returned from the batch:

int exitCode SpringApplication.exit(SpringApplication.run(batchConfiguration, args)); System.exit(exitCode);

Spring MVC, Spring Boot and Resources Caching

Modern web application and browser resources caching

Modern web applications use a large number of static resources such as js files, css, fonts, images, etc.
Even if the internet connections are always more efficient, it’s always worth asking if it’s useful to use the resources caching, that’s to say to allow the browser to store static resources in its cache. In general it’s better to find the right compromise, since in reality resources can also change such as the application js files (very rarely those of the application stack).

Our web applications are based on Spring Boot and Spring MVC which by default do not allow the caching of resources.
To enable resource caching, you can intervene in the application config files. For example, if we want the cache to have a validity of one week,
you have to add the following line in the application.properties file:

spring.resources.cache-period=604800

If a particular resource handler is used, configure it for caching. For example, for resources loaded with the webjars protocol, we had to configure a resource handler for compatibility issues with Websphere. Even in this case just configure the cache period like this:

@Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
super.addResourceHandlers(registry);
registry.addResourceHandler("/webjars/**").addResourceLocations("classpath:/META-INF/resources/webjars/").
setCachePeriod(604800).resourceChain(true).addResolver(new WebJarsResourceResolver(new WasWebJarAssetLocator()));
}

To force the cache to refresh before expiry of the validity period (for example, if you release a new version of the application that requires changes substantial in the js files) two techniques can be used. On the client side, you can force the cache to be refreshed with the classic << Shift + F5 >> from the browser. On the server side you can use the file versioning technique, adding the version in the file name.

Howto migrate LDAP users to Spring Security JDBC

Howto migrate LDAP users to Spring Security JDBC

Lately we are redoing a web portal whose users who have access to the private area of ​​the application are registered through LDAP.
The only request made by the contractor is that the transition to the new portal must be transparent to registered users. This translates into the fact that users do not have to change the password the first time they log in to the new portal. Passwords are stored in the LDAP repository with SSHA (Salted SHA) encoding.
Our application uses Spring Security to manage security and access to the reserved area. Spring Security supports various types of authentication including LDAP itself. As a first idea we thought to use the same LDAP repository already present. After analyzing this solution in detail we have thought not to take this solution for various reasons.
The first is to map the roles related to the permissions of the old application to the roles of our application (feasible but not very nice from the point of functional view). Furthermore, having to maintain two separate servers, an LDAP and a DBMS, when it is possible to have only one DBMS server, is not a good thing from the point of management costs.
So we thought about using the classic Spring Security JDBC authentication. The users will be migrated through a batch that will load the unloading LDAP users (download, for example, done in the csv format) to the JDBC tables. To ensure that password encryption remains the same, just configure Spring Security to use the LdapShaPasswordEncoder class. To do this you need to define the following bean in WebMvcConfiguration:

@Bean
public LdapShaPasswordEncoder passwordEncoderLDAP () {
return new LdapShaPasswordEncoder ();
}

end using it in the AuthenticationManagerBuilder defined in WebSecurityConfiguration like this:

@Autowired
private LdapShaPasswordEncoder ldapPasswordEncoder;

@Override
protected void configure (AuthenticationManagerBuilder auth) throws Exception {
auth.userDetailsService (webJdbcUserDetailsManager).passwordEncoder (ldapPasswordEncoder);
}

in this way the password coding will be the same used by LDAP and for users the transition to the new portal will be transparent.

Oracle, Repeatable Read and JPA

Oracle optimistic and pessimistic lock

Lately we had to solve concurrent management of database access between different threads of the same process or different processes.
The scenario is as follows: web application deployed on websphere, tasks scheduled by spring scheduler within the same web application and standalone external batches. All these threads must be able to work simultaneously on the same data without causing concurrent problems. Above all because the operations involve non-transactional services on Host invoked via CTG. In essence it means to adopt a pessimistic lock approach instead of optimistic lock.

The application stack is as follows:

  • Java 7
  • Spring 3.1
  • JPA 2.0
  • Hibernate 3.6.8
  • Websphere 8.5.5
  • Oracle 12g

Oracle has three types of Isolation Level: Read Uncommitted, Read Committed and Serialization. Missing Repeatable Read that is exactly what would solve our problem without application interventions. Serialization is too strong as an approach, because we do not want to lock the entire table and prevent any insert. We decide to set Read Committed (Oracle default level) but as expected at the first concurrent access we have an optimistic lock problem:

Caused by: javax.persistence.OptimisticLockException: org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction  (or unsaved-value mapping was incorrect)

To solve this type of problems Oracle provides us with the select for update, which means using JPA to lock the instance obtained from the database.
For example, to retrieve objects through a query:

Query q = entityManager().createQuery("SELECT o FROM ObjectA o");
q.setLockMode(LockModeType.PESSIMISTIC_WRITE);

in this way if the object has no lock it will be immediately returned by the db to the calling thread, otherwise the thread will wait until the lock is released.

With this pessimistic lock-oriented approach, lock timeout problems can occur. In this case the exeception should be managed with try / catch to cancel the operation or try again to recover the object. In production it is also possible through an empirical approach to increase the timeout to reduce the frequency of the problem.

Finally, obviously, the retrieve of concurrent objects must be done before invoking non-transactional services such as CTG, to avoid the cancellation of the operation after committing the CTG itself.

GenericConversionService, IdToEntityConverter and StackOverflowError

Spring GenericConversionService, IdToEntityConverter and StackOverflowError

Yesterday, after a code refactoring, doing a test of the web application I get a mythical StackOverflowError:

java.lang.StackOverflowError
at java.util.HashMap.addEntry(HashMap.java:888)
at java.util.LinkedHashMap.addEntry(LinkedHashMap.java:427)
at java.util.HashMap.put(HashMap.java:509)
at java.util.HashSet.add(HashSet.java:217)
at org.springframework.core.convert.support.GenericConversionService.addInterfaceHierarchy(GenericConversionService.java:452)
at org.springframework.core.convert.support.GenericConversionService.getMatchingConverterForTarget(GenericConversionService.java:437)
at org.springframework.core.convert.support.GenericConversionService.findConverterForClassPair(GenericConversionService.java:349)
at org.springframework.core.convert.support.GenericConversionService.getConverter(GenericConversionService.java:244)
at org.springframework.core.convert.support.GenericConversionService.canConvert(GenericConversionService.java:145)
at org.springframework.core.convert.support.IdToEntityConverter.matches(IdToEntityConverter.java:54)
...

strange … on a web form that had not been subject to code changes for a long time.

The project is based on Spring 3.1.0 and Java 7.

By debugging, I find that the failed conversion affects the domain class Address to which the static findAddress method has been added. The signature is as follows:

public static Address findAddress(Address address)

the method has been added to prevent duplicate persistance addresses on the db.

So now the Address class has two static finders:

public static Address findAddress(Address address)
public static Address findAddress(Long id)

the IdToEntityConverter class uses the getFinder method whose rules are (from javadoc):

the finder method must be public, static, have the signature find[EntityName]([IdType]), and return an instance of the desired entity type.

The problem is that IdType is converted by searching its finder, which is always of the same type Address, causes the loop resulting in StackOverflowError.

The solution is to rename

findAddress(Address address)

method in

retrieveAddress(Adress address).

Howto install web application with Spring Roo 2.0 on Websphere 8.5.5

Install web application with Spring Roo 2.0.0.RC1 on Websphere 8.5.5.12

To install spring roo 2.0.0.RC1 on Websphere 8.5.5.12 you have to solve some problems. The environment is as follows:

  • Ubuntu 15.04 (GNU/Linux 3.19.0-15-generic x86_64)
  • IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 20170419_34
  • Spring Roo 2.0.0.RC1
  • Websphere 8.5.5.12

JPA problem

The first problem is the JPA compatibility. Roo 2.0.0.RC1 uses jpa 2.1 while WAS 8.5.5.12 jpa 2.0.At the start of the application there is such a mistake:

caused by: java.lang.ClassCastException: com.ibm.websphere.persistence.PersistenceProviderImpl incompatible with javax.persistence.spi.PersistenceProvider at javax.persistence.Persistence$1.isLoaded(Persistence.java:110) ~[hibernate-jpa-2.1-api-1.0.0.Final.jar:1.0.0.Final]

To fix this problem, you need to add the HibernatePersistenceProviderResolver class to your project:

HibernatePersistenceProviderResolver.java

and register it in the Application class in the onStartup method

@Override public void onStartup(ServletContext servletContext) throws ServletException {
HibernatePersistenceProviderResolver.register();
super.onStartup(servletContext); }

XML-APIS problem

The second problem is due to xml-apis jar:

Caused by: java.lang.ClassCastException: org.apache.xalan.processor.TransformerFactoryImpl incompatible with javax.xml.transform.TransformerFactory at javax.xml.transform.TransformerFactory.newInstance(Unknown Source) ~[xml-apis-1.4.01.jar:na]

to solve it you need to make an exclusion in pom.xml like this:

<dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-data-jpa</artifactId>
      <exclusions>
      	<exclusion> 
          <groupId>xml-apis</groupId>
          <artifactId>xml-apis</artifactId>
        </exclusion>        
      </exclusions>
    </dependency>

WEBJARS problem

Third issue concerns the webjars protocol:

Caused by: java.lang.NullPointerException at java.io.FilterInputStream.close(FilterInputStream.java:192) at sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream.close(JarURLConnection.java:121) at org.springframework.web.context.support.ServletContextResource.isReadable(ServletContextResource.java:120)

For some reason WAS uses the wsjar protocol instead of jar. Also to access the resources within the jars WAS must have a trailing slash in the path.To solve this, you need to add the following class:

WasWebJarAssetLocator.java

and log it in to WebMvcConfiguration in the addResourceHandlers method

@Override public void addResourceHandlers(ResourceHandlerRegistry registry) { super.addResourceHandlers(registry); registry.addResourceHandler("/webjars/**").addResourceLocations("classpath:/META-INF/resources/webjars/"). resourceChain(false).addResolver(new WebJarsResourceResolver(new WasWebJarAssetLocator())); }

finally adding variable to was (web container custom properties)

com.ibm.ws.webcontainer.SkipMetaInfResourcesProcessing = true

Transactions problem

Another issue with regard to transactions is:

Caused by: java.lang.LinkageError: com/ibm/websphere/uow/UOWSynchronizationRegistry.registerInterposedSynchronization(Ljavax/transaction/Synchronization;)V (loaded from file:/opt/IBM/WebSphere/AppServer/plugins/com.ibm.ws.runtime.jar by org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader@d30c8d80) called from class org.springframework.transaction.jta.WebSphereUowTransactionManager$UOWActionAdapter (loaded from file:/opt/IBM/WebSphere/AppServer/profiles/AppSrv01/installedApps/UBDSK01Node01Cell/uman-1_0_1_war.ear/uman-1.0.1.war/WEB-INF/lib/spring-tx-4.3.3.RELEASE.jar by com.ibm.ws.classloader.CompoundClassLoader@19d0788f[war:uman-1_0_1_war/uman-1.0.1.war]   Local ClassPath: com.ibm.ws.uow.embeddable.EmbeddableUOWManagerImpl.runUnderNewUOW(EmbeddableUOWManagerImpl.java:791) ~[com.ibm.ws.runtime.jar:na] ... 112 common frames omitted

to solve it you have to make some exclusions in the pom. For javax.transaction we make the exclusion and scope provided:

<dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-data-jpa</artifactId>
      <exclusions>
      	<exclusion> 
          <groupId>xml-apis</groupId>
          <artifactId>xml-apis</artifactId>
        </exclusion>
        <exclusion> 
          <groupId>javax.transaction</groupId>
          <artifactId>javax.transaction-api</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
<dependency>
      <groupId>javax.transaction</groupId>
      <artifactId>javax.transaction-api</artifactId>
      <scope>provided</scope>
    </dependency>

for jta we make the exclusion:

<dependency>
      <groupId>io.springlets</groupId>
      <artifactId>springlets-boot-starter-web</artifactId>
      <exclusions>
      	<exclusion>
      		<groupId>javax.transaction</groupId>
    		<artifactId>jta</artifactId>
      	</exclusion>
      </exclusions>
    </dependency>

that’s all.

Having done all this, the application will run smoothly.