Websphere, SQL Server and Deadlock

How avoid deadlock with Websphere and SQL Server

When you create a datasource in Websphere for a SQL Server database, the default isolation level is Repeatable Read. At first it seems the best choice, but repeatable read as isolation level means the presence of table lock, and with table lock, a deadlock can happen at any time!

For example, considere this application stack:

Java 8
Spring Boot 2.4.1
Websphere 8.5.5
SQL Server 2008

If you need a transaction with REQUIRES_NEW propagation, a deadlock can happen if you are going to update the same tables in the parent and child transaction. This because the isolation level is Repeatable Read.

The solution is to relax the isolation level to Read Committed. You avoid deadlock and with hibernate optimistic lock, there is no possibility to miss information.

To set the isolation level of a datasource in Websphere, you have to navigate the path:

Data sources > [datasource name] > Custom properties

and set

webSphereDefaultIsolationLevel=2

where 2 means Read Committed.

That’s what you need to avoid deadlock!

And remember, if you need a pessimistic lock, with JPA is simple to set table lock for a single query (see for example my previous post about it).

Advertisement

Fixefid 3.0.0 released!

Fixefid 3.0.0 released!

The Fixefid 3.0.0 has been released.

Fixefid is a Java library for working with flat fixed formatted text files and CSV files.

A lot of improvements has been carried out:

  • Validation: now it’s possibile to obtain the validation info of all fields. That’s simplify the error management. For instance, if something goes wrong during the processing of the input string, no exception is thrown, and it’s possibile to obtain the validation error info of all fields.
  • Annotation: all configurations can be managed with annotations. The extended properties style has been maintained but only for advanced configuration, like call back needs.
  • Valid values accepted list in field annotation
  • Enum style: it’s been deprecated and not longer supported
  • Minor bug fixes and enhancements

Fixefid Home Page

Fixefid Javadoc

Fixefid Maven Repository

Fixefid 2.0.0 released!

Fixefid 2.0.0 released!

I’m very proud to announce Fixefid 2.0.0 has been released!

Fixefid is a Java library for working with flat fixed formatted text files and CSV files.

The version 2.0.0 includes:

  • Java 8
  • CSV Record
  • Annotations for record and field extended properties
  • Fields Occurrences and subOrdinal
  • Simple Boolean Format
  • Logging
  • Minor bug fixes and enhancements

Fixefid Home Page

Fixefid Javadoc

Fixefid Maven Repository

Using Spring Security 5 to integrate with OAuth2 secured RESTFull API without login and servlet context.

Howto config a Spring Security OAuth2 client that is capable of operating outside of the context of a HttpServletRequest,
e.g. in a scheduled/background thread and/or in the service-tier.

With the new Spring Security 5, there are a lot of examples about howto configure a client to access service like, Facebook, GitHub and many others with the standard OAuth2.
But today I found diffulties to get documentations about howto access OAuth2 secured RESTFull API with a RestTemplate client, without login and servlet context. And so I had to debug Spring Security framework to figure out the right configuration.

My env is as a follow:

Java 8
Spring Boot 2.4.1
Spring Security 5.4.2
Spring Web 5.3.2

The pom dependencies are:

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-oauth2-client</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-security</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

The api that I have to access are secured with OAuth2 with GRANT_TYPE=PASSWORD and client auth method equals to POST.
Every call to the Api must contains an Authorization header with the access token of type Bearer.
To obtain the access token, we need a token uri, a client id and the client username/password. All the information must be provided by the resource server.

What we need is a RestTemplateConfig. The file of this example can be found here.

We’re going to see the config step by step. First of all we need a ClientRegistrationRepository

    Builder b = ClientRegistration.withRegistrationId(registrationId);
    b.authorizationGrantType(AuthorizationGrantType.PASSWORD);
    b.clientAuthenticationMethod(ClientAuthenticationMethod.POST);
    b.tokenUri(tokenUri);
    b.clientId(clientId);
    ClientRegistrationRepository clients = new InMemoryClientRegistrationRepository(b.build());

the tokenUri end the clientId must be provided by the resource server.

Then we need the service:

    OAuth2AuthorizedClientService service = new InMemoryOAuth2AuthorizedClientService(clients);

the authorized client provider:

    OAuth2AuthorizedClientProvider authorizedClientProvider = OAuth2AuthorizedClientProviderBuilder.builder().password().refreshToken().build();

and the manager:

    AuthorizedClientServiceOAuth2AuthorizedClientManager manager = new AuthorizedClientServiceOAuth2AuthorizedClientManager(
            clients, service);
    manager.setAuthorizedClientProvider(authorizedClientProvider);
    manager.setContextAttributesMapper(new Function<OAuth2AuthorizeRequest, Map<String, Object>>() {

        @Override
        public Map<String, Object> apply(OAuth2AuthorizeRequest authorizeRequest) {
            Map<String, Object> contextAttributes = new HashMap<>();
            String scope = authorizeRequest.getAttribute(OAuth2ParameterNames.SCOPE);
            if (StringUtils.hasText(scope)) {
                contextAttributes.put(OAuth2AuthorizationContext.REQUEST_SCOPE_ATTRIBUTE_NAME,
                        StringUtils.delimitedListToStringArray(scope, " "));
            }

            String username = authorizeRequest.getAttribute(OAuth2ParameterNames.USERNAME);
            if (StringUtils.hasText(username)) {
                contextAttributes.put(OAuth2AuthorizationContext.USERNAME_ATTRIBUTE_NAME, username);
            }

            String password = authorizeRequest.getAttribute(OAuth2ParameterNames.PASSWORD);
            if (StringUtils.hasText(password)) {
                contextAttributes.put(OAuth2AuthorizationContext.PASSWORD_ATTRIBUTE_NAME, password);
            }

            return contextAttributes;
        }

    });

then we can add an interceptors to the RestTemplate:

@Bean
public RestTemplate restTemplate(RestTemplateBuilder builder, OAuth2AuthorizedClientManager manager) {
    RestTemplate restTemplate = builder.build();
    restTemplate.getInterceptors().add(new BearerTokenInterceptor(manager, username, password, registrationId));

    return restTemplate;
}


public class BearerTokenInterceptor implements ClientHttpRequestInterceptor {
    private final Logger LOG = LoggerFactory.getLogger(BearerTokenInterceptor.class);

    private OAuth2AuthorizedClientManager manager;
    private String username;
    private String password;
    private String registrationId;

    public BearerTokenInterceptor(OAuth2AuthorizedClientManager manager, String username, String password, String registrationId) {
        this.manager = manager;
        this.username = username; 
        this.password = password;
        this.registrationId = registrationId; 
    }

    @Override
    public ClientHttpResponse intercept(HttpRequest request, byte[] bytes, ClientHttpRequestExecution execution)
            throws IOException {
        String accessToken = null;
        OAuth2AuthorizedClient client = manager.authorize(OAuth2AuthorizeRequest.withClientRegistrationId(registrationId)
                .attribute(OAuth2ParameterNames.USERNAME, username)
                .attribute(OAuth2ParameterNames.PASSWORD, password)
                .principal(principal).build()); 
        accessToken = client.getAccessToken() != null ? client.getAccessToken().getTokenValue() : null;
        if (accessToken != null) {
            LOG.debug("Request body: {}", new String(bytes, StandardCharsets.UTF_8));
            request.getHeaders().add("Authorization", "Bearer " + accessToken);
            return execution.execute(request, bytes);
        } else {
            throw new IllegalStateException("Can't access the API without an access token");
        }
    }

}

before every call, the manager try to authorize the client with the username and password provided by the resource server. If the authentication is successful, the server return a json like this:

{
“access_token”: “hjdhjYU00jjTYYT….”,
“token_type”: “Bearer”,
“expires_in”: “3600”,
“refresh_token”: “hdshTT55jhds…”,
}

Since the server support refresh token, we have configured the authorizedClientProvider to manage the refresh token in case the access token provided is expired.

That’s all!

Converting fixed fields text record to JSON

Using a REST Service for converting fixed fields text record to json with Fixedfid java library.

Converting fixed fields text record to JSON can be realized in many ways. A solution can be the using of a REST Service combined with the Fixefid java library.

The environment is as a follows:

  • Java 8
  • Spring Boot 2.3.4.RELEASE
  • Spring Web
  • Fixefid 1.1.0
  • Spring Doc Openapi 1.5.0

The Fixefid java library permits to define a fixed fields text record with Java Bean or Java Enum. In this case the definition by Java Bean can be used to annotate a resource representation class of a REST Service.

For instance, we want converting a customer record like this one:

String record = "0000000000000000001Paul                                              Robinson                                          ";

to a json object like this one:

{
"id": 1,        
"firstName": "Paul",        
"lastName": "Robinson"    
}

To model the customer representation, we can create a resource representation class:

@FixefidRecord
public class Customer {
	@FixefidField(fieldLen = 19, fieldOrdinal = 0, fieldType = FieldType.N)
	private Long id;
@FixefidField(fieldLen = 50, fieldOrdinal = 1, fieldType = FieldType.AN)
private String firstName;

@FixefidField(fieldLen = 50, fieldOrdinal = 2, fieldType = FieldType.AN)
private String lastName;

protected Customer() {
}

public Customer(String firstName, String lastName) {
    this.firstName = firstName;
    this.lastName = lastName;
}

@Override
public String toString() {
    return String.format("Customer[id=%d, firstName='%s', lastName='%s']", id, firstName, lastName);
}

public Long getId() {
    return id;
}

public String getFirstName() {
    return firstName;
}

public String getLastName() {
    return lastName;
}
}

The resource representation class is annotated with the Fixefid annotations. Then we can create the record request:

public class RecordRequest {
	private Long requestId;
	private String record;

public String getRecord() {
    return record;
}

public void setRecord(String record) {
    this.record = record;
}

public Long getRequestId() {
    return requestId;
}

public void setRequestId(Long requestId) {
    this.requestId = requestId;
}
}

and the Customer response:

public class CustomerResponse {
	private Long requestId;
	private Long responseId;
	private Customer customer;

public CustomerResponse(Long requestId, Long responseId, Customer customer) {
    this.requestId = requestId;
    this.responseId = responseId;
    this.customer = customer;
}

public Long getRequestId() {
    return requestId;
}
public void setRequestId(Long requestId) {
    this.requestId = requestId;
}
public Long getResponseId() {
    return responseId;
}
public void setResponseId(Long responseId) {
    this.responseId = responseId;
}
public Customer getCustomer() {
    return customer;
}
public void setCustomer(Customer customer) {
    this.customer = customer;
}
}

last, the rest controller:

@RestController
public class CustomerController {
	private final AtomicLong counter = new AtomicLong();

        @PostMapping(path = "/recordtocustomer", consumes = 
        "application/json", produces = "application/json")
        public CustomerResponse recordToCustomer(@RequestBody RecordRequest 
           request) {
        Customer customer = new Customer(null, null);
        new BeanRecord(customer, request.getRecord());
        return new CustomerResponse(request.getRequestId(), 
        counter.incrementAndGet(), customer);
     }
}

With Postman we can test the service:

Here the project of the example on github.

Howto deal with fixed fields text record and JPA Entity

If the persistance layer is realized with JPA, we can mapping record fields directly to Entity fields

Fixefid is a java library wich permits to define a fixed fields text record with Java Bean or Java Enum. Often a text record must be retrieved from data persisted on a database. Or a text record must be persisted on a database.

The solution is to define a mapping from the record’s fields with the persistence model. If the persistance layer is realized with JPA Entities, the mapping can be done directly to the JPA Entity with the Fixefid annotations. Infact a JPA Entity is a POJO, that’s a Java Bean. And so, we can annotate the JPA Entity with the Fixefid annotations to realize the mapping, without the need to create two models, one for the record and another one for the JPA Entity.

The environment is as a follows:

  • Java 8
  • Spring Boot 2.3.4.RELEASE
  • Spring Data JPA
  • Fixefid 1.1.0
  • H2 Database

For example we can have a Customer bean like this one:

@Entity
@FixefidRecord
public class Customer {
	@Id
	@GeneratedValue(strategy = GenerationType.AUTO)
	@FixefidField(fieldLen = 19, fieldOrdinal = 0, fieldType = FieldType.N)
	private Long id;
	
	@FixefidField(fieldLen = 50, fieldOrdinal = 1, fieldType = FieldType.AN)
	private String firstName;
	
	@FixefidField(fieldLen = 50, fieldOrdinal = 2, fieldType = FieldType.AN)
	private String lastName;

	protected Customer() {
	}

	public Customer(String firstName, String lastName) {
		this.firstName = firstName;
		this.lastName = lastName;
	}

	@Override
	public String toString() {
		return String.format("Customer[id=%d, firstName='%s', lastName='%s']", id, firstName, lastName);
	}

	public Long getId() {
		return id;
	}

	public String getFirstName() {
		return firstName;
	}

	public String getLastName() {
		return lastName;
	}
}

The Customer above is annotated with Entity and FixefidRecord. The fields are annotated with FixefidField and other JPA annotations. To obtain the record from the database:

Customer customer = repository.findById(1L);
String record = new BeanRecord(customer).toString();

To save the record to the database:

String newRecord = "0000000000000000001Paul                                              Robinson                                          ";
Customer newCustomer = new Customer();
new BeanRecord(newCustomer, newRecord);
repository.save(newCustomer);

I made a video tutorial where the example above is explained in the detail way.

Here the project of the example on github.

Howto deal with fixed fields text record without substring

Howto avoid substring approach to deal with fixed fields formatted text records. The solution is the Java library Fixefid

Often we have to develop Java methods to deal with fixed fields formatted text records. The format is often used nowadays yet to files transfer between mainframe and unix machines. Almost always the approach is using substring like this:

String address = record.substring(10, 20);
String location = record.substring(20, 30);

But that is an error prone pattern. The records are length thousands of chars with many many fields and make a mistake with the offset is not so rarely. And moreover when you have to format the fields to create the record, you have to deal with string right padding, zero lead numeric left padding, boolean format, date format, decimal format, decimal separator, etc etc…

The solution is using a Java library like Fixefid, wich permits to start from a specification in excel, model the record with Java Bean or Java Enum and deal with fields by getter, setter and toString Java methods.

I’ve made a video tutorial where I explain in the detail way the process to start from an excel like this:

to create a Java Bean model like this:

to obtain a records txt file like this:

In this way we can using the standard java bean development to deal with fixed fields formatted text files.

For example you can create a Java Bean like this:

@FixefidRecord(recordLen = 600)
public class Student extends Person {
@FixefidField(fieldLen = 10, fieldOrdinal = 8, fieldType = FieldType.N,       
         fieldMandatory = FieldMandatory.INOUT)
private Long studentId;
….
}

and create a BeanRecord like this:

Student student = createStudent();
BeanRecord studentRecord = new BeanRecord(student, null, null, 
       createStudentMapFieldExtendedProperties());
studentRecord.toNormalize();
String studentRecordAsString = studentRecord.toString();
System.out.println("Student Record=[" + studentRecordAsString + "]");

The video tutorial explains the whole process in the detail way.

Here the java project developed on the video tutorial.

If you don’t want to use Java Bean, the same record can be modeled with Java Enum.

Where are my java projects?

The question is: what eclipse I used to import a java project?

After years and years of tests, developments, new projects, new versions of eclipse, I’m in the situation of opening the folder in which all my projects are present but not remembering which version of eclipse they were imported with. And so the other day I asked myself, how do I know, given a java project, which version of eclipse  did I use? This without opening eclipse … obviously. The answer is as follows: for each eclipse workspace, just look in the folder

${eclipse_workspace}\.metadata\.plugins\org.eclipse.core.resources\.projects

Got it!