John's Technical Blog

In one of the projects I am working on, I was tasked from taking a Tomcat application that ran on an EC2 server, and get it running with the Spring Boot framework in a Docker container. The twist is that the application needed read-write access to the resources folder within the project. In order to do that, the application needed to be run as an "exploded" WAR file (It had been run that way on the old server as well).

I accomplished this with this Docker file:

The WAR file is compiled with the "spring-boot-maven-plugin", and includes all the dependency JARs, so the application can be run stand-alone.

That entry in the POM is:

And the "org.springframework.boot.loader.WarLauncher" Class is what Spring Boot uses to bootstrap the applicatiton on the embedded Tomcat.

I am working on updating applications at work to Java 9. I have been using OpenPojo for years to test all my POJOs in one go. However, I found that the tests started throwing the exception:

java.lang.NoClassDefFoundError: Could not initialize class com.openpojo.reflection.java.packageloader.Package

I traced through the OpenPojo code and found that it was hard coded to read the Java class path from the old "sun.boot.class.path" system property. This property has been completely removed from Java 9, in favor of "java.class.path," which has been available since at least Java 7. (See https://docs.oracle.com/javase/8/docs/technotes/tools/windows/findingclasses.html)

I submitted my findings in an issue for the developers consideration: https://github.com/oshoukry/openpojo/issues/108

In the meantime, I developed the following workaround that can be inserted into a current POJO test class, and will allow the code to function the same way in Java 9.

In some of my load testing, I want to run a certain number of transactions. My script takes parameters for a number of threads and the number of loops all the threads should go to. When I have a certain target number I want to get to, and an idea of what number of threads I want, I would like to figure out exactly how many loops I need to run for an exact number of threads.

For example, lets say I want to process 16,000,000 transactions with about 300 threads. I can run the following Java class, and figure out that if I use 320 threads, I need 50,000 loops.

Enter an integer: 16000000
Even divisors for 16000000
8000000 x 2 = 16000000
4000000 x 4 = 16000000
3200000 x 5 = 16000000
2000000 x 8 = 16000000
1600000 x 10 = 16000000
1000000 x 16 = 16000000
800000 x 20 = 16000000
640000 x 25 = 16000000
500000 x 32 = 16000000
400000 x 40 = 16000000
320000 x 50 = 16000000
250000 x 64 = 16000000
200000 x 80 = 16000000
160000 x 100 = 16000000
128000 x 125 = 16000000
125000 x 128 = 16000000
100000 x 160 = 16000000
80000 x 200 = 16000000
64000 x 250 = 16000000
62500 x 256 = 16000000
50000 x 320 = 16000000
40000 x 400 = 16000000
32000 x 500 = 16000000
31250 x 512 = 16000000
25600 x 625 = 16000000
25000 x 640 = 16000000
20000 x 800 = 16000000
16000 x 1000 = 16000000
15625 x 1024 = 16000000
12800 x 1250 = 16000000
12500 x 1280 = 16000000
10000 x 1600 = 16000000
8000 x 2000 = 16000000
6400 x 2500 = 16000000
6250 x 2560 = 16000000
5120 x 3125 = 16000000
5000 x 3200 = 16000000
4000 x 4000 = 16000000
3200 x 5000 = 16000000
3125 x 5120 = 16000000
2560 x 6250 = 16000000
2500 x 6400 = 16000000
2000 x 8000 = 16000000
1600 x 10000 = 16000000
1280 x 12500 = 16000000
1250 x 12800 = 16000000
1024 x 15625 = 16000000
1000 x 16000 = 16000000
800 x 20000 = 16000000
640 x 25000 = 16000000
625 x 25600 = 16000000
512 x 31250 = 16000000
500 x 32000 = 16000000
400 x 40000 = 16000000
320 x 50000 = 16000000
256 x 62500 = 16000000
250 x 64000 = 16000000
200 x 80000 = 16000000
160 x 100000 = 16000000
128 x 125000 = 16000000
125 x 128000 = 16000000
100 x 160000 = 16000000
80 x 200000 = 16000000
64 x 250000 = 16000000
50 x 320000 = 16000000
40 x 400000 = 16000000
32 x 500000 = 16000000
25 x 640000 = 16000000
20 x 800000 = 16000000
16 x 1000000 = 16000000
10 x 1600000 = 16000000
8 x 2000000 = 16000000
5 x 3200000 = 16000000
4 x 4000000 = 16000000
2 x 8000000 = 16000000
1 x 16000000 = 16000000
=== End ===

Essentially, this provides all the numbers that evenly divide the provided number. Or, in other words, find multipliers that will result in a desired product.

Over the course of the last couple of years, I have checked out dozens of projects with IntelliJ IDEA from various Git repositories. Many of these projects were one-time checkouts, and a slew of them are no longer valid because of domain name changes.


There is no internal mechanism to remove URLs from this dialogue box, and I had not found anything on the Internet on how to do this. I was certain that this information had to be in a configuration file somewhere.

I ran across this page: Directories used by the IDE, which headed me in the correct direction. I eventually found the file I was looking for (using Mac OS X):

~/Library/Preferences/<PRODUCT><VERSION>/options/vcs.xml

This file has a set of <UrlAndUserName> elements that can be individually deleted, as needed:


<application>
  <component name="GitRememberedInputs">
    <option name="visitedUrls">
      <list>
        …
        <UrlAndUserName>
          <option name="url" value="https://github.com/appium/sample-code.git" />
          <option name="userName" value="" />
        </UrlAndUserName>
        …
      </list>
    </option>
    <option name="cloneParentDir" value="$USER_HOME$/IdeaProjects" />
  </component>
</application>
 
Exit IntelliJ IDEA completely, and start it up again. The next time you use the Clone Repository dialogue box (e.g., using "Check out from Version Control" in the Welcome dialogue box), you will see the list reduced to whatever entries you left in the vcs.xml file.

This solution was tested in version 2017.1, and I confirmed the same file location for version 2016.1.

When setting up Readiness and Liveness checks in Kubernetes for Kafka connectors, the use of the httpGet described in my previous blog post (Kubernetes Readiness and Liveness with Spring Boot Actuator) is not an option because there is no endpoint to reference. These can be deployed with the Apache Kafka REST Proxy, which gets us on the right path, but doesn't quite work how we want in this respect.

The Kafka REST Proxy provides endpoints that allow one to get some basic status info about connectors. However, the standard Kubernetes httpGet calls use status code >= 200 and < 400 to determine the status, and since the Kafka REST status endpoint always provides a 200 status code, it is not possible to use this methodology to determine if a connector is down.

What we would like to do is check the content of the status call, and do a string comparison. For example, when the service is up, the status endpoint indicates that the state is "RUNNING":

# curl http://10.30.128.1:8083/connectors/mysql-kafka-connector/status

{"name":"mysql-kafka-connector","connector":{"state":"RUNNING","worker_id":"10.30.128.1:8083"},"tasks":[{"state":"RUNNING","id":0,"worker_id":"10.30.128.1:8083"}]}

We can pause the connector using this endpoint:

# curl -i -X PUT http://10.30.128.1:8083/connectors/mysql-kafka-connector/pause

HTTP/1.1 202 Accepted

And then the state is changed to PAUSED:

# curl http://10.30.128.1:8083/connectors/mysql-kafka-connector/status

{"name":"mysql-kafka-connector","connector":{"state":"PAUSED","worker_id":"10.30.128.1:8083"},"tasks":[{"state":"PAUSED","id":0,"worker_id":"10.30.128.1:8083"}]}


To accomplish this check, we can leverage the exec command probe:

readinessProbe: 

  exec: 

    command:

      - /bin/sh 

      - -c 

      - curl -s http://127.0.0.1:8083/connectors/mysql-kafka-connector/status | grep "RUNNING"

  initialDelaySeconds: 240

  periodSeconds: 5

  timeoutSeconds: 5 

  successThreshold: 1

  failureThreshold: 10 

livenessProbe:

  exec: 

    command: 

      - /bin/sh

      - -c 

      - curl -s http://127.0.0.1:8083/connectors/mysql-kafka-connector/status | grep "RUNNING"

  initialDelaySeconds: 300 

  periodSeconds: 60

  timeoutSeconds: 10 

  successThreshold: 1

  failureThreshold: 3

The the exec command allows us to execute a shell command. In this case:

• running the shell (/bin/sh)
• telling it to run a single command (-c)
• with the command being a cURL call to the specific connector status endpoint, and grepping for the string "RUNNING"

When the grep is successful, Kubernetes interprets this as a success. If the grep comes back empty (i.e., "RUNNING" is not found), then it gets viewed as a failure. You can then test this on the server by pausing the service in question as described above.

To get the service running again and start passing readiness and liveness again, then you will want to use the RESUME endpoint.

# curl -i -X PUT http://10.30.128.1:8083/connectors/mysql-kafka-connector/pause

HTTP/1.1 202 Accepted