A few months ago I wrote about my simple test to see how both Bard and ChatGPT 3.5 did with a simple programmer interview question (summary: neither one passed) and since Bard has had its innards recently replaced (with Google Gemini LLM) which is supposedly an improvement over its original PaLM LLM AI, I thought I’d rerun the test for Bard to see if it did any better.

The prompt I submitted back in September was:

create java 8 code/class that provides a macro replacement capability. specifically a string can contain zero or more macro references where a reference is something like “$name” (the $ char introduces the alphanumeric macro name), and the processing of the string requires a map and each $name is replaced with the corresponding value in the map. one special case is that if the $ char is encountered within a single-quote string, then it’s not taken as a macro

and so I submitted it exactly as above to the improved Bard. This is what is produced (though I did add the import java.util.* and the main() method):

import java.util.*;

public class MacroReplacer {

    private final Map<String, String> macros;

    public MacroReplacer(Map<String, String> macros) {
        this.macros = macros;
    }

    public String replaceMacros(String text) {
        StringBuilder builder = new StringBuilder();
        int pointer = 0;
        while (pointer < text.length()) {
            int dollarIndex = text.indexOf('$', pointer);
            if (dollarIndex == -1) {
                builder.append(text.substring(pointer));
                break;
            }

            // Check if dollar is inside a single-quote string
            int prevQuoteIndex = -1;
            int quoteIndex = text.indexOf('\'', pointer);
            while (quoteIndex != -1 && quoteIndex < dollarIndex) {
                prevQuoteIndex = quoteIndex;
                quoteIndex = text.indexOf('\'', quoteIndex + 1);
            }
            if (prevQuoteIndex != -1 && prevQuoteIndex % 2 == 1) {
                // Dollar inside single-quote string, append as-is
                builder.append(text.substring(pointer, dollarIndex + 1));
                pointer = dollarIndex + 1;
                continue;
            }

            // Extract macro name
            int nextSpaceIndex = text.indexOf(' ', dollarIndex + 1);
            int macroNameLength = nextSpaceIndex != -1 ? nextSpaceIndex - dollarIndex - 1 : text.length() - dollarIndex - 1;
            String macroName = text.substring(dollarIndex + 1, dollarIndex + 1 + macroNameLength);

            // Replace macro with value from map
            String macroValue = macros.get(macroName);
            if (macroValue != null) {
                builder.append(macroValue);
            } else {
                builder.append('$');
                builder.append(macroName);
            }

            pointer = dollarIndex + 1 + macroNameLength;
        }
        return builder.toString();
    }

    /**
     * had to add (copied from ChatGPT's code)
     */
    public static void main(String... args)
    {
        Map<String, String> macroMap = Map.of(
                "name", "John",
                "age", "30"
        );
        MacroReplacer bard = new MacroReplacer(macroMap);

        String input = "Hello $name! You are $age years old. Do not replace '   $age' in single quotes.";
        String output = bard.replaceMacros(input);

        System.out.println(output);
    }
}

and when I ran it I got

$name!30$age' in single quotes.

which, sadly, is much worse than original Bard.

Maybe GPT4 is better (I don’t want to pay to test that) but, then again, maybe I’m justified in still not worrying about them taking my job.