How to use ChatGPT to write code - and my favorite trick to debug what it generates

4 days ago 3

One of the more intriguing discoveries about ChatGPT is that it can write pretty good code. I first tested this out in 2023 when I asked it to write a WordPress plugin my wife could use on her website. ChatGPT did a fine job, but it was a simple project.

So, how can you use ChatGPT to write code as part of your daily coding practice? Here's a quick summary:

ChatGPT can produce both useful and unusable code. For best results, provide clear and detailed prompts.
ChatGPT excels in assisting with specific coding tasks or routines, rather than building complete applications from scratch.
Use ChatGPT to find and choose the right coding libraries for specific purposes, and engage in an interactive discussion to narrow your options.
Be cautious about who owns AI-generated code and always verify the code's reliability. Don't blindly trust the generated output.
Treat interactions with ChatGPT as a conversation. Refine your questions based on the AI's responses to get closer to the desired output.

Now, let's explore ChatGPT in considerably more depth.

What types of coding can ChatGPT do well?

There are two important facts about ChatGPT and coding. First, the AI can write useful code.

The second is that the AI can get completely lost, fall into a rabbit hole, chase its tail, and produce unusable garbage.

Also: The best AI for coding in 2025 (and what not to use)

I found this fact out the hard way. After I finished the WordPress plugin for my wife, I decided to see how far ChatGPT could go.

I wrote a very careful prompt for a Mac application, including detailed descriptions of user interface elements, interactions, what would be provided in settings, how they would work, and more. Then, I fed the prompt to ChatGPT.

ChatGPT responded with a flood of text and code. Then, it stopped mid-code. When I asked the AI to continue, it vomited even more code and text. I requested continue after continue, and it dumped out more and more code. However, none of the output was usable. The AI didn't identify where the code should go, how to construct the project, and -- when I looked carefully at the code produced -- it left out major operations I requested, leaving in simple text descriptions stating "program logic goes here".

Also: How ChatGPT scanned 170k lines of code in seconds and saved me hours of work

After repeated tests, it became clear that if you ask ChatGPT to deliver a complete application, the tool will fail. A corollary to this observation is that if you know nothing about coding and want ChatGPT to build something, it will fail.

Where ChatGPT succeeds -- and does so very well -- is in helping someone who already knows how to code to build specific routines and get tasks done. Don't ask for an app that runs on the menu bar. But if you ask ChatGPT for a routine to put a menu on the menu bar, and paste that into your project, the tool will do quite well.

Also, remember that, while ChatGPT appears to have a tremendous amount of domain-specific knowledge (and often does), it lacks wisdom. As such, the tool may be able to write code, but it won't be able to write code containing the nuances for specific or complex problems that require deep experience.

Also: How to use ChatGPT to create an app

Use ChatGPT to demo techniques, write small algorithms, and produce subroutines. You can even get ChatGPT to help you break down a bigger project into chunks, and then you can ask it to help you code those chunks.

So, with that in mind, let's look at some specific steps for how ChatGPT can help you write code.

How to use ChatGPT to write code

This first step is to decide what you will ask of ChatGPT -- but not yet ask it anything. Decide what you want your function or routine to do, or what you want to learn to incorporate into your code. Decide on the parameters you'll pass into your code and what you want to get out. And then look at how you're going to describe it.

Also: How to write better ChatGPT prompts

Imagine you're paying a human programmer to do this task. Are you giving that person enough information to be able to work on your assignment? Or are you too vague and the person you're paying is more likely to ask questions or turn in something entirely unrelated to what you want?

Here's an example. Let's say I want to be able to summarize any web page. I want to feed the AI this article and get back a well-considered and appropriate summary. As my input, I'll specify a web page URL. As my output, it's a block of text with a summary.

Continuing with the example above, an old school way of extracting web page data was to find the text between HTML paragraph tags.

However, with the rise of AI tools, you can use an AI library to do an intelligent extract and summary. One of the places ChatGPT excels (and it's also an area you can easily verify to avoid its authoritative-but-wrong behavior pattern) is finding libraries and resources.

Also: The best free AI courses

OpenAI (the maker of ChatGPT) sells API access to its LLMs to do exactly what we want. But in the case of this example, let's assume we don't want to pay transaction fees.

So, let's look at interacting with ChatGPT to figure out how to use such a tool, for free, with a project that runs in PHP.

I started with a prompt to elicit information about what libraries would provide the desired functionality. A library (for those reading along who aren't programmers) is a body of code a programmer can access that does a lot of the heavy lifting for a specific purpose. A big part of modern programming is finding and choosing the right libraries, so this is a good starting point.

In this case, I'm looking at blocks of code written by other people that will summarize text. Here's my first prompt:

Describe ten different open source AI libraries (and the languages they work with) that I can use to generate a summary of the main core contents of any web page, ignoring any ads or embedded materials.

This prompt gave me exactly what I wanted, including a mention of OpenAI's offerings. I think OpenAI would do great here, but for this hypothetical project, I don't want to budget for API fees. So, I'll narrow down the question:

Are any of these free?

ChatGPT hedged its bets with its answer: "Yes, all ten of these AI libraries are open source and free to use. However, some of them may have usage limits or require payment for access to additional features or resources." So, based on that response, I clarified my query:

Which of these libraries have no usage limits and don't require any additional payment or licensing?

Notice how this is very much a conversation. I don't have to re-ask the original question. I'm just drilling down as I might if I had an expert next to me. This time, ChatGPT gave me eight library choices, but none mentioned the PHP language I was planning to use to code. So, here's the next prompt:

Of those 8 libraries, can I use any with PHP?

It returned three libraries, but I wasn't sure about what each did. So, another question:

What's the difference between Sumy, Gensim, and NLTK?

I still wasn't sure, so I clarified my use plan and then asked:

If I want to create summaries of web page news articles, which library would work better?

The answer was clear and promising: "Sumy is specifically designed for text summarization, which is the task of creating a summary that captures the most important information from a piece of text." So, it was time to see what was involved in using Sumy with PHP. I asked my last question for this part of the project:

Can you explain how to use Sumy from PHP?

Feel free to play along on your computer and paste these prompts into ChatGPT. Notice that, in step one, I decided what program module to get help on. Then, in this step, I had a conversation with ChatGPT to decide what library to use and how to integrate it into my project.

Also: The best AI chatbots

That approach might not seem like programming, but I assure you it is. Programming isn't just blasting lines of code onto a page. Programming is figuring out how to integrate all the various resources and systems, and how to talk to all the components of your solution. Here, ChatGPT helped me do that integration analysis.

By the way, I was curious whether Google's Gemini AI could help similarly. Gemini did give some extra insights into the planning aspect of programming over ChatGPT's responses.

So, don't hesitate to use multiple tools to triangulate your answers. Here's that story: Gemini vs. ChatGPT: Can Gemini help you code? Since I wrote that article, Google added some coding capabilities to Gemini, but they're not all that great. You can read about that capability here: I tested Google Gemini's new coding skills. It didn't go well. And even more recently, I dug into Gemini Advanced. The AI is still not passing many tests.

Also: How I test an AI chatbot's coding ability - and you can too

Coding is next.

OK, let's pause here. This article is entitled "How to use ChatGPT to write code." And it will. But what we're really doing is asking ChatGPT to write example code.

Also: The rise and fall in programming languages' popularity since 2016 - and what it tells us

Let's be clear. Unless you're writing a small function (like the line sorter/randomizer ChatGPT wrote for my wife), ChatGPT can't write your final code. First, you'll have to maintain it. ChatGPT is terrible at modifying already-written code. Terrible, as in, it doesn't do it. So, to get fresh code, you have to ask ChatGPT to generate something new. As I found previously, even if your prompt is virtually identical, ChatGPT may unexpectedly change what it gives you.

So, bottom line: ChatGPT can't maintain your code, or even tweak it.

That limitation means you have to do the legwork yourself. As we know, the first draft of a piece of code is rarely the final code. So, even if you expect ChatGPT to generate final code, it would be a starting point, and one where you need to take it to completion, integrate it into your bigger project, test it, refine it, debug it, and so on.

But that issue doesn't mean the example code is worthless -- far from it. Let's look at a prompt I wrote based on the project I described earlier. Here's the first part:

Wite a PHP function called summarize_article.

As input, summarize_article will be passed a URL to an article on a news-related site like ZDNET.com or Reuters.com.

I'm telling ChatGPT the programming language it should use. I'm also telling the AI the input and providing two sites as samples to help ChatGPT understand the article style. Honestly, I'm not sure ChatGPT didn't ignore that bit of guidance. Next, I'll tell it how to do the bulk of the work:

Inside summarize_article, retrieve the contents of the web page at the URL provided. Using the library Sumy from within PHP and any other libraries necessary, extract the main body of the article, ignoring any ads or embedded materials, and summarize it to approximately 50 words. Make sure the summary consists of complete sentences. You can go above the 50 words to finish the last sentence, if necessary.

This approach is very similar to how I'd instruct an employee. I'd want that person to know that they weren't only restricted to Sumy. If they needed another tool, I wanted them to use it.

Also: IBM will train you in AI fundamentals for free, and give you a skill credential - in 10 hours

I also specified an approximate number of words to create bounds for what I wanted as a summary. A later version of the routine might take that number as a parameter. I then ended by saying what I wanted as a result:

Once processing is complete, code summarize_article so it returns the summary in plain text.

The resulting code is pretty simple. ChatGPT called on another library (Goose) to retrieve the article contents. It then passed that summary to Sumy with a 50-word limit and returned the result. But once the basics are written, it's a mere matter of programming to go back in and add tweaks, customize what's passed to the two libraries, and deliver the results:

One interesting point of note. When I originally tried this test in early 2023, ChatGPT created a sample call to the routine it wrote, using a URL from after 2021. At that time, in March 2023, ChatGPT's dataset only went to 2021. Now, the ChatGPT knowledge base extends to the end of June 2024 and can search the web. But my point is that ChatGPT made up a sample link that it couldn't possibly know about:

https://www.reuters.com/business/retail-consumer/teslas-musk-says-fremont-california-factory-may-be-sold-chip-shortage-bites-2022-03-18/

I checked that URL against Reuters' site and the Wayback Machine, and it doesn't exist. Never assume ChatGPT is accurate. Always double-check everything it gives you.

I showed you a few ways that ChatGPT makes mistakes or hallucinates. All programmers make mistakes, even the AI ones.

But you can do several things to help refine your code, debug problems, and anticipate errors that might crop up. My favorite new AI-enabled trick is to feed code to a different ChatGPT session (or a different chatbot entirely) and ask, "What's wrong with this code?"

Inevitably, something comes up. The AI sometimes identifies edge cases or error checks that should be added to the code, or situations that might break if a confluence of unlikely events should occur. I've then coded around those error conditions, making code more robust.

Does ChatGPT replace programmers?

Not now -- or, at least -- not yet. ChatGPT programs at the level of a talented first-year programming student, but it's lazy (like that first-year student). The tool might reduce the need for entry-level programmers.

However, at its current level, I think AI will make life easier for entry-level programmers (and even programmers with more experience) to write code and look up information. It's a time-saver, but the AI can't do many programming tasks by itself -- at least now. In 2030? Who knows.

How do I get coding answers in ChatGPT?

Just ask it. You saw above how I used an interactive discussion dialog to narrow the answers. Don't expect one question to do all your work magically. But use the AI as a helper and resource, and it will give you a lot of helpful information.

Also: Want a programming job? Learn these three languages

Of course, test that information -- because, as John Schulman, a co-founder of OpenAI, said: "Our biggest concern was around factuality, because the model likes to fabricate things."

Is the code generated by ChatGPT guaranteed to be error-free?

Hell, no! But you also can't trust the code human programmers write. I certainly don't trust any code I write. Code comes out of the code-making process incredibly flawed. There are always bugs. Before you ship, you need to test, test, and test again. Then, alpha test with a few chosen victims. Then beta test with your wider user community.

Even after all that work, there will be bugs. Just because an AI plays at this coding thing doesn't mean it can do bug-free code. Do not trust. Always verify. And you still won't have fully bug-free output. Such is the nature of the universe.

What do I do if the code I get back is wrong?

I recommend considering the chatbot as a slightly uncooperative student or subordinate employee. What would you do if that person gave you back code that didn't work? You'd send them back out with instructions to do it again and get it right. That's about what you should do with ChatGPT (I've tested this with ChatGPT 4 and 4o). When things don't work, I say: "That didn't work. Please try again."

Also: Google's AI podcast tool transforms your text into stunningly lifelike audio - for free

The AI does just that. It often gives me back different variations on the same problem. I've repeated this process four or five times on occasion until I've gotten a working answer. Sometimes, though, the AI runs out of ideas. Other times, the try-again answer is completely (and I do mean completely) unrelated to what you've requested.

When it becomes apparent you've reached the edge of the AI's ability to remain sane on the problem, you'll have to buckle up and code it yourself. But 9 times out of 10, especially with basic coding or interface-writing challenges, the AI does its job successfully.

How detailed should my description of a programming issue be when asking ChatGPT?

Detailed. The more you leave open for interpretation, the more the AI will go its own way. When I give prompts to ChatGPT to help me while programming, I imagine I'm assigning a programming task to one of my students or someone who works for me.

Also: 6 ways to write better ChatGPT prompts - and get the results you want faster

Did I give that person enough details to create a first draft or will that person have to ask me additional questions? Worse, will that person have so little guidance that they'll go off in entirely the wrong direction? Don't be lazy here. ChatGPT can save you hours or even days of programming (it has for me), but only if you give it useful instructions to begin with.

If I use ChatGPT to write my code, who owns it?

As it turns out, there's not a lot of case law yet to answer this question. The US, Canada, and the UK require something copyrighted to have been created by human hands, so code generated by an AI tool may not be copyrightable. There are also issues of liability based on where the training code came from and how the resulting code is used.

ZDNET did a deep dive on this topic, spoke to legal experts, and produced three articles. If you're concerned about this issue (and if you're using AI to help with code, you should be), I recommend you read them:

What programming languages does ChatGPT know?

The answer is most languages. I tested common modern languages, like PHP, Python, Java, Kotlin, Swift, C#, and more. But then I had the tool write code in obscure dark-age languages like COBOL, Fortran, Forth, LISP, ALGOL, RPG (the report program generator, not the role-playing game), and even IBM/360 assembly language.

As the icing on the cake, I gave it this prompt:

Write a sequence that displays 'Hello, world' in ascii blinking lights on the front panel of a PDP 8/e

The PDP 8/e was my first computer, and ChatGPT gave me instructions to toggle in a program using front-panel switches. I was impressed, gleeful, and ever so slightly afraid.

Can ChatGPT help me with data analysis and visualization tasks?

Yes, and a lot of it can be done without code. Check out my entire article on this topic: The moment I realized ChatGPT Plus was a game-changer for my business.

I also did a piece on generated charts and tables: How to use ChatGPT to make charts and tables.

But here's where it gets fun. In the article above, I asked ChatGPT Plus, "Make a bar chart of the top five cities in the world by population," and it did. But do you want code? Try asking:

Make a bar chart of the top five cities in the world by population in Swift. Pull the population data from online. Be sure to include any necessary libraries.

By adding "in Swift" you're specifying the programming language. By specifying where the data comes from and forcing ChatGPT Plus to include libraries, the AI brings in the other resources the program needs. That's why, fundamentally, programming with an AI's help requires you to know things about programming. But if you do, it's cool, because three sentences can get you a chunk of annotated code. Nice, huh?

How does ChatGPT handle differences between dialects and implementations?

We don't have exact details on this issue from OpenAI, but our understanding of how ChatGPT is trained can shed some light on this question. Remember that dialects and implementations of programming languages (and their little quirks) change much more rapidly than the full language. This reality makes it harder for ChatGPT (and many programming professionals) to keep up.

Also: How I used ChatGPT to write a custom JavaScript bookmarklet

As such, I'd work off these two assumptions:

The more recent the dialectic change, the less likely ChatGPT knows about it, and
The more popular a language, the more training data it's learned from and, therefore, the more accurate it will be.

What's the bottom line? ChatGPT can be a helpful tool. Just don't ascribe superpowers to it. Yet.

You can follow my day-to-day project updates on social media. Be sure to follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.

Read Entire Article