ChatGPT is an artificial intelligence-based (AI) chatbot developed by OpenAI, built on top of the organization’s GPT-3.5 and GPT-4 large language models (LLM).

This blog post details how to write Java code to connect to OpenAI’s REST API. Once you start, you can add more functionality to your code.

Creating your account with OpenAI

1. Create an account on https://platform.openai.com.
2. Under Menu => Personal, select “View API keys.”
   

   Figure 1. OpenAI’s API key generation screen.

3. Create and save your API key for later use.
4. On the left-side Menu, select Organization =>Settings, and request that an “Organization ID” be generated. Copy this for later usage.

Figure 2. OpenAI’s setup screen to create an Organization ID.

Java code

You’ll need to create three files:

1. FileHelper.java

   import java.io.File;
   import java.io.FileWriter;
   import java.io.IOException;
   import java.nio.charset.StandardCharsets;
   import java.nio.file.Files;
   import java.nio.file.Paths;
   import java.util.LinkedList;
   import java.util.List;
   public class FileHelper {
   
   public static String readLinesAsString(File file) {
   List<String> returnLines = new LinkedList<String>();
   String text = "";
   try {
   text = new String(Files.readAllBytes(Paths.get(file.getAbsolutePath())),
   StandardCharsets.UTF_8);
   
   } catch (IOException e) {
   e.printStackTrace();
   }
   return text;
   }
   
   public static List<String> readLines(File file) {
   List<String> returnLines = new LinkedList<String>();
   try {
   String text = new String(Files.readAllBytes(Paths.get(file.getAbsolutePath())),
   StandardCharsets.UTF_8);
   
   String[] lines = text.split("\n");
   for (String line : lines) {
   
   returnLines.add(line);
   }
   
   } catch (IOException e) {
   e.printStackTrace();
   
   }
   return returnLines;
   }
   
   public static void writeStringToFile(String line, File file) {
   try {
   FileWriter myWriter = null;
   if (file.exists()) {
   myWriter = new FileWriter(file, true);//if file exists append to file. Works fine.
   } else {
   System.out.println("Could not find the file " + file + ". Creating it again");
   
   file.createNewFile();
   myWriter = new FileWriter(file);
   }
   myWriter.write(line);
   myWriter.close();
   // System.out.println("Successfully wrote to the file. "+file.getAbsolutePath());
   } catch (IOException e) {
   e.printStackTrace();
   System.out.println("An error occurred in writing to file " + file + " e=" + e);
   
   }
   }
   }

2. Request.json

   {
   "model": "text-davinci-003",
   "prompt": "Which was the best selling car 2021 in USA?",
   "max_tokens": 64,
   "temperature": 0.5
   }

3. ChatGPTAPIExample.java

   import java.io.BufferedReader;
   import java.io.File;
   import java.io.InputStreamReader;
   import java.io.OutputStream;
   import java.net.HttpURLConnection;
   import java.net.URL;
   import java.nio.charset.StandardCharsets;
   
   public class ChatGPTAPIExample {
   public static void listTokens() {
   
   try {
   
   //This API will fetch the models available.
   URL url = new URL("https://api.openai.com/v1/models");
   HttpURLConnection con = (HttpURLConnection) url.openConnection();
   con.setRequestMethod("GET");
   con.setRequestProperty("Content-Type", "application/json");
   con.setRequestProperty("Accept", "application/json");
   //Make sure you put the right Organization key saved earlier.
   con.setRequestProperty("OpenAI-Organization", "your-org-key");
   con.setDoOutput(true);
   //Make sure you put the right API Key saved earlier.
   con.setRequestProperty("Authorization", "Bearer your-api-key");
   
   int responseCode = con.getResponseCode();
   System.out.println("Response Code : " + responseCode);
   
   BufferedReader in = new BufferedReader(
   new InputStreamReader(con.getInputStream()));
   String inputLine;
   StringBuffer response = new StringBuffer();
   
   while ((inputLine = in.readLine()) != null) {
   response.append(inputLine);
   }
   in.close();
   
   System.out.println(response);
   } catch (Exception e) {
   System.out.println(e.getMessage());
   }
   
   }
   
   public static void prompts() {
   
   try {
   
   URL url = new URL("https://api.openai.com/v1/completions");
   HttpURLConnection con = (HttpURLConnection) url.openConnection();
   con.setRequestMethod("POST");
   con.setRequestProperty("Content-Type", "application/json");
   con.setRequestProperty("Accept", "application/json");
   //Make sure you put the right Organization key saved earlier.
   con.setRequestProperty("OpenAI-Organization", "your-org-key");
   con.setDoOutput(true);
   //Make sure you put the right API Key saved earlier.
   con.setRequestProperty("Authorization", "Bearer your-api-key");
   
   //Make sure to relace the path of the json file created earlier.
   String jsonInputString = FileHelper.readLinesAsString(new File("C:\\yourpath\\request.json"));
   
   try (OutputStream os = con.getOutputStream()) {
   byte[] input = jsonInputString.getBytes(StandardCharsets.UTF_8);
   os.write(input, 0, input.length);
   }
   
   
   
   int responseCode = con.getResponseCode();
   System.out.println("Response Code : " + responseCode);
   
   BufferedReader in = new BufferedReader(
   new InputStreamReader(con.getInputStream()));
   String inputLine;
   StringBuffer response = new StringBuffer();
   
   while ((inputLine = in.readLine()) != null) {
   response.append(inputLine);
   }
   in.close();
   
   System.out.println(response);
   } catch (Exception e) {
   System.out.println(e.getMessage());
   }
   
   }
   
   public static void main(String[] args) {
   listTokens();
   prompts();
   }
   }

Compile and run them once you have the files created. You can build and run in a way that’s most comfortable for you. For example, even using the command line should work, as follows:

javac *.java
java ChatGPTAPIExample

Sample output
Response Code : 200

{“id”:”cmpl-6wFzNtira0jiFrmXHOUkjZpqytNca”,”object”:”text_completion”,”created”:1679342505,”model”:”text-davinci-003″,”choices”:[{“text”:”\n\nIt is too early to tell which car will be the best-selling car in 2022 in the USA.”,”index”:0,”logprobs”:null,”finish_reason”:”stop”}],”usage”:{“prompt_tokens”:10,”completion_tokens”:22,”total_tokens”:32}}

Conclusion

Now that you’ve learned how to get started connecting an application to OpenAI’s API and can integrate it with other applications, you can take advantage of more of ChatGPT’s capabilities. For more details, please refer to https://platform.openai.com/docs/api-reference.

Debugging Java Virtual Machine (JVM) crashes is a common troubleshooting task for Java developers. In this blog, I’ll explain the methodology and process you can use to troubleshoot an OutOfMemory (OOM) exception due to a thread leak to help you easily find the root cause of Java crashes.

Why do I get OOMs?

OOM errors are not new to a Java developer. They’re Java’s way of telling us that there is insufficient memory to allocate an object.  There are many flavors of OOMs that one can experience while working with a Java program. The most common occurs when Java heap space runs out. There are other flavors like exceeding the garbage collection (GC) overhead limit or swap space, etc. You might also get an OOM error while creating a new thread. Here are some possible OOM scenarios.

• If there are multiple Java processes running on the server, process A could be consuming a large thread count while process B complains about this error in the log. (That B process is also often responsible for generating the large thread leaks.)
• Operating system (OS) open file handles have been exceeded, or the process permitted for the logged in user has been exceeded.
• A thread leak is causing a memory shortage at the server, which will cause the JVM process to throw out an OOM error.
• A possible thread leak in a third-party library or product.

Note: If it is a VM, make sure that the server you are working in has the right resources assigned, especially when it’s in a shared cluster and other VMs are consuming more memory than required.

Recipe for troubleshooting thread leaks

Every application flow and environment will be different, so troubleshooting actions will also vary accordingly. That said, our process for troubleshooting and confirming the culprit code for a thread leak is to:

1. Set up the server for troubleshooting
2. Identify the suspect process causing thread leaks
3. Generate a thread dump and heap dump
4. Narrow down the leaky code

Here is a sample code that can be deployed on the server to generate the OOM:

import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;

public class OutOfMemorySimulate {

	public static void main(String[] args) {

		OutOfMemorySimulate simulate = new OutOfMemorySimulate();
		
		while(true)
		simulate.execute();
	}

	private void execute() {

		System.out.println("Processing the data.");
		ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(4);
		for (int i = 0; i < 10; i++) {
			MyThread task = new MyThread();
			System.out.println("A new task has been added : " + task.getName());
			executor.execute(task);
		}
		System.out.println("Process");
	}
}

Next, run this the command:

java OutOfMemorySimulate -Xmx100m  -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/simulate.hprof

Follow the steps below to reach the culprit code.

Step 1: Set up the server for troubleshooting

There are a few things you need to do to check up on the server, as follows:

• Since the server may not have a Java Development Kit (JDK) set up, copy the JDK .tar file to your server
• Extract the Java .tar file with the following code:
  • $cd /tmp
  • $tar -xvf jdk-11.0.10+9.tar

Step 2: Identify the suspect process

Now that you’re ready to explore and troubleshoot, let’s try to find out something about our process, starting with more descriptive output:

$ ps -aef | grep java

user     41     1  7 Jun05 ?   02:31:08 java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath= -Xmx100m

Next, let’s check how much memory the process is consuming:

$ top
  PID USER      PR  NI    VIRT    RES    SHR   S  %CPU %MEM     TIME+ COMMAND
   41user         20   0      4.9g    1.6g  26220 S    8.6       5.9         151:21.34 java

Note that although we have set the -Xmx JVM parameter to 100m, the resident memory usage kept increasing, as shown highlighted in yellow above.

This indicates that:

1. The Java process heap memory is not occupied, so the program is not inadvertently allocating objects into heap space.
2. The OS still has a memory shortage.
3. The OS has denied the allocation of more memory to the Java process.
4. This could be the thread leak.

If you check the logs, then you should also notice an exception similar to below:

Exception in thread “main” java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:803)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:937)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1343)
        at OutOfMemorySimulate.execute(OutOfMemorySimulate.java:21)
        at OutOfMemorySimulate.main(OutOfMemorySimulate.java:11)

So, the OOM does seem to be related to the thread leak. It’s time to check the thread dump.

Step 3: Generate thread dump

Let’s look at the various ways to create a thread dump of the process on the server.

JDK provides a jstack utility, which can attach to a specific Java process and print stack traces of all threads, including native stack frames. To do this, you will need the Java process ID (PID) from the ps -ef command (for UNIX) explained earlier.

$ jstack -l <PID>

This prints the full thread dump. At the top of the thread dump (or by searching within the dump file), you will be able to find the thread counts.

Threads class SMR info:
_java_thread_list=0x0000012b82c90830, length=46723, elements={
0x0000012bc020d000, 0x0000012bda8d7000, 0x0000012bda8e0800, 0x0000012bda931000,
0x0000012bda932000, 0x0000012bdb231800, 0x0000012bdb23c000, 0x0000012bdb25d800,
0x0000012bdb3bd000, 0x0000012bdb3c8000, 0x0000012bdb3da000, 0x0000012bdb3db000,
0x0000012bdb3db800, 0x0000012bdb3dc800, 0x0000012bdb3e5800, 0x0000012bdb3e8000,
0x0000012bdb3e9000, 0x0000012bdb3e9800, 0x0000012bdb3ea800, 0x0000012bdb3eb800,

As you can see in the highlighted text in the figure above, there are many (45,000+) threads within the thread pool, and that number will increase continuously.

The stack trace for the threads in the WAITING state would have the following pattern:

"pool-11679-thread-1" #46725 prio=5 os_prio=0 cpu=0.00ms elapsed=1.76s tid=0x0000012b82b72000 nid=0xaabc waiting on condition  [0x0000002e006fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.7/Native Method)
	- parking to wait for  <0x00000000fe543518> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.7/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.7/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take(java.base@11.0.7/LinkedBlockingQueue.java:433)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.7/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.7/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.7/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.7/Thread.java:834)

There are other ways to obtain the thread dump. You can use any proprietary tool, such as the YourKit Java Profiler or the kill -3 <PID> command in UNIX, etc.

Step 4: Narrow down the leaky code

To confirm our analysis further, we will also generate a heap dump, which will help us reach the exact location of the code that is causing the thread leaks. You can do this a few ways:

1. Use the heap dump that is generated after the Java process has crashed. Remember that you need to specify the -XX:+HeapDumpOnOutOfMemoryError as part of the Java command starting the Java process.
2. Specify the path of the storage where there is sufficient disk space available:
   -XX:HeapDumpPath.
3. In your server, make sure you have automation in place to copy the HPROF to a persistent location or provide the path to the persistent storage.
4. Generate the heap dump manually by tracking the process memory where usage is high.

Let’s use JMap to take a heap dump of the Java process:

$ jmap -dump:live,format=b,file=/tmp/high-threads.hprof <PID>

You can use different visual tools like JVisualizer, JMat, etc. to analyze the heap dump. The following is an example of how a heap dump analyzer can help you find the problematic code.

In this case, you can see that the thread object is taking up the top position with 52,000+ objects. The call traces will also confirm the line numbers to look for. The heap dump and thread dump analysis lead us to a condition where root cause are:

• Thread pool executors are used to create threads but executor.shutdown() is not called to release the thread objects.
• The thread pool executor should have been invoked at the initialization stage and not during the business process method, which is called during each transaction process.
• The Max Thread pool size is not defined, leading to spawning of new threads every time.

Conclusion

That’s just one example of our learnings while tracking down these tricky thread leakages that lead to continuous Java process crashes. We’d also recommend that you

• Conduct regular scans and set alerts on your application level when it generates and OutOfMemory exception.
• Use BMC Log TrueSight Monitoring and BMC TrueSight Generic JVM Monitoring to monitor your application logs.
• Do thread dump and heap dump analysis of your application to identify the thread leaks as part of the development process.
• For ad hoc/quick monitoring of your environments, use OS commands to count the threads being used for the process

ls  /proc/pid/task|wc -l
lsof -a -p pid |wc -l

• Set a limit on the maximum number of threads in the pool as a best coding practice while using scheduled thread pool execution.
• Keep self-monitoring turned on to monitor the maximum thread count of your process and troubleshoot and take recovery actions if it is crossing the threshold.
• At the OS level, make sure you have the “core file size to unlimited” set to the appropriate value ( ulimit –c) to let it generate the memory dump HPROF when an accident happens.
• Refer to the memory tuning explained on the link https://docs.bmc.com/docs/tsps113/troubleshooting-java-memory-management-935083760.html

Better performance and scalability of an application do not always depend on the amount of memory allocated. There are often occasions where allocating more memory has resulted in an adverse situation. This blog explores the adverse impact of performance degradation and explains how to create a balance while allocating memory to Java applications.

Enterprise applications sizing background

For this exercise, we will refer to our in-house IT operations product, which is being used for the end-to-end operations management of our own IT business unit as well as multiple enterprise-level customers. It’s important to understand the volumetrics that the solution will be monitoring. This will help us reverse engineer and map the numbers to our BMC-recommended system requirements, which are documented as part of the sizing and scalability guidelines. Once this mapping is established, it becomes less complicated to size the IT operations stack.

You can find additional information on presentation server sizing and scalability here and information on infrastructure management server sizing and scalability here.

Does the process need extra memory?

Once deployed, we should make it a practice to monitor the health of the stack. This will help us to understand whether any modifications are needed to meet changing business requirements. In addition, any performance degradation in the stack will proactively trigger an alarm, making it easier for us to remediate before end users are impacted.

Different layers of monitoring include:

1. Monitoring the logs from the application
2. Operating system monitoring
3. Java Virtual Machine (JVM) monitoring

The Java Development Kit (JDK) comes bundled with VisualVM, which helps us to get a holistic view of the running JVMs, including overall memory consumption and the respective threads associated with the same.

The above analysis will help us investigate further and may result in enhancing or reducing current resources allocated to the stack. We would need to map the findings according to the sizing documents referenced above and look for discrepancies, based on the specific process or thread that we are analyzing.

Can you increase the memory allocation for processes directly?

The answer is NO. We should always be mindful of making changes to the resource in the running stack. The reason is there are multiple entities tightly coupled together in the stack (e.g., it’s not a standalone single-layer application), so resource changes to one entity will negatively or positively impact the related entities, leading to new issues in the stack and further degrading the performance of the application.

Example of garbage collection setting which worked for an application

Below are the Java 8 parameters that we would normally use, especially while tuning garbage collection. This is applicable to the Garbage-First Garbage Collector (G1 GC).

• XX:+DisableExplicitGC
• XX:+G1NewSizePercent
• XX:+MaxGCPauseMillis
• XX:+XX:MaxMetaspaceSize
• XX:+MaxMetaspaceSize
• XX:+UseCompressedOops
• XX:+UseStringDeduplication
• XX:+UseG1GC

With respect to the operations management application, we made changes based on our observation for the G1 GC parameters. Below are the properties that we considered before and after making the changes.

Before making the changes:

From G1 GC:

• Option=XX:+UseG1 GC
• Option=XX:MaxGCPauseMillis=200
• Option=XX:ParallelGCThreads=20
• Option=XX:ConcGCThreads=5
• Option=XX:InitiatingHeapOccupancyPercent=70

After making the changes to the parallel GC:

• Option=Dsun.rmi.dgc.client.gcInterval=3600000
• Option=Dsun.rmi.dgc.server.gcInterval=3600000

Here, we ran the collection every hour and performed a parallel garbage collection (GC). This helped us to reduce the CPU footprints while the G1 GC was executed. Overall process memory usage is also controlled with fixed-interval GC cycle runs.

This may not work correctly if we don’t have proper heap settings. If the setting is very low, then the GC may be invoked before the above hourly interval, running automatically instead of when we want it to run. Normally, increasing the max heap by a factor of 20 percent is a good start to confirm whether the GC is being invoked every hour.

There have been instances where the application indicates that the process is running out of memory but internally the respective JVM is not using that much memory. In this case, the application process’s JVM needs a max heap allocation, but due to limited resource availability, the OS could not release the max to the JVM process. This results in an out-of-memory error due to incorrect heap settings and insufficient RAM available to the VM—it’s not a JVM process error.

Normally, we would see an exception similar to java.lang.OutOfMemoryError: unable to create new native thread, which indicates that Java demanded the memory chunk but there was insufficient RAM available on the server. In this case, adding extra heap space will not help.

In these kinds of scenarios where the overall RAM of the machine is not properly allocated, the role of GC becomes critical. In addition, if the GC cycles are not run properly, this leads to the piling of objects in the heap memory with both direct and indirect references. It can also take more processing CPU time/performance to do the cleanup when the GC executes.

Most of these would be inactive, or not-live, objects, but an inadequate or infrequent GC cycle leads to unnecessary heap consumption with these objects. This kind of issue leads us to modify some of the properties as shown above.

Below is a snapshot where the JVM required more memory, but it had reached max heap.

Figure 1. JVM reaches max heap.

The following shows the potential to go OutOfMemory (OOM) because of heap.

Figure 2. JVM nearing OutOfMemory and crashes.

Using BMC products to monitor these JVM processes, we get a memory graph that indicates that the JVM had 32 GB RAM, all of which has been utilized, so any further requests by the JVM processes cannot be handled by the OS and the JVM process crashes, as shown above.

Figure 3. JVM utilizing 32 GB of memory.

The above illustration shows that increasing the JVM heap does not always help to mitigate the OOM situation. There could be additional factors like how much RAM is available on the VM and whether the OS has enough memory to release to the JVM process.

We had another instance from a customer where one of the JVM processes was running out of memory. The end impact was the application crashed, generating a memory dump file.

Figure 4. Snapshot from the memory dump.

The above stack showed where the OOM happened; drilling down more, we could see the actual issue.

Figure 5. The operation that triggered the issue.

This is again another scenario where an end user would be prompted to increase the JVM process allocation, which may resolve the problem for a couple of days, but it will eventually crash with the same error.
In this case, we had to handle this specific issue through code optimization.

Does the server have the right memory allocated?

Most virtual machine (VM) servers have shared resources. When there is a sudden chunk of memory needed for a server, there should not be scarcity in the VM pool.

Let’s keep in mind that CPU and memory are proprietary to the nodes or the VM itself, where the application is installed. Even before we install any specific application, we allocate CPU and memory to the VM.

Within the application, it’s up to the vendor (application developer) to determine how the CPU and memory would be allocated for the seamless performance of the solution. This is where, based on our sizing recommendation, we allocate memory to the running JVM processes.

But how do we make sure these allocations at the VM level are the best possible numbers we could imagine? Well, there is no straightforward answer to this, as this depends on monitoring the VM nodes using capabilities both inside and outside the solution.

On the IT operations solution end, BMC has come up with a knowledge module called the VMware vSphere Knowledge Modules (VSM KM). This specific KM is used to monitor the entire virtual center (VC) where our application is running, with respect to memory and CPU. These are metrics that reveal the health of the VC.

Figure 6. CPU utilization metric.

Figure 7. CPU ready time.

Figure 8. Memory balloon.

Figure 9. Memory used.

Using the virtual center monitoring tool

Monitoring the VC will help us to understand scenarios where the solution itself is down and we don’t have the built-in capability for VC monitoring. Based on our experience, we have isolated a few metrics and a performance chart, which help us to understand the overall health of the VC, as follows.

ESX overprovisioned

• Memory overcommitment
• CPU overcommitment

Memory ballooning

• Should not happen if host is performing as per expectation

Storage latency

• Response of the command sent to the device, desired time within 15-20 milliseconds

VC dashboard

• This will alert us to any open alarms regarding the health of the VC and respective VMs

Datastore

• The datastore of the VM where our IT operations is installed; should be in a healthy condition with sufficient space available

Performance chart

• Verify the performance chart for memory and CPU from the VC level

How these help

These monitoring metrics help us to identify the potential impact on the overall performance of the IT operations solution when proper CPU and memory are not allocated within the application, dependent on their availability to the VM nodes themselves.

Conclusion

It’s very important to monitor and understand how much CPU and memory we are allocating to the VM nodes so we can adjust to the lowest level possible. At the highest level, they will affect garbage collection performance.

For more information on this topic, please see our documentation on hardware requirements to support small, medium, and large environments here.