Spring Boot Performance: Automated Tests to Detect API Calls Within Database Transactions

We should not make API calls within Database Transactions to increase performance and security. This post is going to talk about:

• Why API calls should not be made within Database Transaction?
• How to validate API calls are not made within Database Transaction?

This post describes a test-first approach to finding the problem. The next post will discuss fixing it. By the end of this post, you’ll have a repeatable automated test that can detect this case. In the next blog post, I will discuss how to separate API calls from database transactions and different patterns that I have encountered.

Remember: the syntax will change but you can use the same principles in the database of your choice.

Why API calls should not be made within Database Transaction?

The main reason is the database connection remains open for a long time. This can cause the following problems:

Pool Exhausted

• Database and ORMs want to work efficiently and have a limited connection pool size.
• When a part of the code wants to make a DB call, it will acquire a connection and release it when it’s done using it.
• By having API calls within Database transaction the connection can remain open for much much longer and can consume all the connections in the connection pool.
• When this happens, you could get a Pool Exhausted exception.
• Therefore, it is important to only use database connections for a short time so that it is free to be used by other parts of the code.

Dead Lock and Timeout

• Whenever we open a transaction we open a context in the database and some cases we lock on particular database rows.
• When this happens, other database transactions won’t have access to that row. This can cause timeouts or in the worst case a deadlock.

Both of these are security concerns as well because they can exploit the availability of the service.

If your application is small you may not think of these things but as the application grows these performance and security concerns become important.

How to validate API calls are not made within Database Transaction?

It is possible to validate that API calls are not within Database transactions right from our Unit Tests and Integration Tests with simple test constructs without modification to any of our current tests.

In both cases the idea is the following:

1. intercept when an API call is made
2. check-in at that moment if the call is made within the transaction
3. if yes, throw an exception

Note while the below example is in Java, you can replicate it in any language.

Integration Test

In Spring Boot, we can intercept WebClient by adding a filter to it. And then adding a code to check it’s called within Transactional or TransactionTemplate

@TestConfiguration
@Order(1)
@RequiredArgsConstructor
public class DatabaseTransactionApiCallBlockerConfig {
    private final WebClient.Builder httpClientBuilder;

    @PostConstruct
    public void enforceTransactionSafetyOnHttpCalls() {
        httpClientBuilder.filter(((request, next) -> {
            if (TransactionSynchronizationManager.isSynchronizationActive()) {
                throw new RuntimeException("Performance Exception: API call attempted during DB transaction");
            }
            return next.exchange(request);
        }));
    }
}

The following is the hibernate code which returns will tell us if the call is made with transactional.

TransactionSynchronizationManager.isSynchronizationActive()

Unit Test

This was the most exciting part of solving this problem. We want to ensure a good test pyramid and have more number of unit tests that integration tests to ensure faster tests. But if we aren’t using an actual database then how can we tell if the call is made within transaction?

By a similar actual actually, that is to check if the caller is within a transaction. And this we can do by checking the StackTrace.

If the caller of the transaction is in a code construct related to the transaction, we throw an exception.

In the case of Spring Boot, we want to ensure,

• The calling method is not annotated with @Transactional
• The calling method is class name is not TransactionTemplate.
  • If you have a custom wrapper, then you may also detect your wrapper class name here.

Here’s how you can do it

• create a new mock factory: mockApi
• this would contain an interceptor that checks if any method in the call stack is transaction-related

@NoArgsConstructor(access = AccessLevel.PRIVATE)
public class MockitoApiInterceptor implements InvocationListener {
    public static <M> M mockApi(Class<M> clazz) {
        return Mockito.mock(clazz, Mockito.withSettings()
            .invocationListeners(new MockitoApiInterceptor()));
    }

    @SneakyThrows
    static boolean isTransactionOpen() {
        return Arrays.stream(Thread.currentThread().getStackTrace())
            .anyMatch(element ->
                element.toString().contains(TransactionTemplate.class.getName()) // programmatic tx
                || findMethod(element)
                    .map(m -> m.getAnnotation(Transactional.class) != null) // @Transactional check
                    .orElse(false) // skip if method is private
            );
    }

    static Optional<Method> findMethod(StackTraceElement element) {
        Class<?> clazz = ClassLoader.getSystemClassLoader().loadClass(element.getClassName());
        return Arrays.stream(clazz.getDeclaredMethods())
            .filter(method -> method.getName().equals(element.getMethodName()))
            .findFirst();
    }

    @Override
    public <T> T answer(InvocationOnMock invocation) throws Throwable {
        if (isTransactionOpen()) {
            throw new RuntimeException("Performance Exception: API call attempted during DB transaction");
        }
        return invocation.callRealMethod();
    }
}

Usage

@ExtendWith(MockitoExtension.class)
class MovieServiceTest {
    // Before
    @Mock
    private MovieApiClient movieApiClient;

    // After
    @Spy
    private MovieApiClient movieApiClient = mockApi(MovieApiClient.class); // guarded spy; statically imported

    ... tests
}

Chaos engineering (Optional)

Do you want to be adventurous, you can enable the Bean for IntegrationTest bean configuration in your DEV or 🙃 PROD for real-world chaos engineering.

Conclusion

It’s important that API calls are executed outside Database Transactions to ensure that the Database isn’t locked for a long time. If the database stays locked for a long time, the number of open connections ends and it will lead to an error.

We can easily validate if API calls are not made within database transactions in both the Integration test and Unit tests.

The next blogpost is about how to not to make API calls within transaction. You can also checkout my TDD series for more such validations.

Question: Have you encountered other hidden performance or transactional issues in your codebase? I’d love to hear and learn from your experience.