1). My Prompt: “is 5923 a prime number?”

2). The AI tool’s initial response:

Let’s carefully check if 5,923 is a prime number.

---

Step 1: Check divisibility by small primes

• 2: 5,923 is odd → not divisible by 2.
• 3: Sum of digits = 5 + 9 + 2 + 3 = 19 → 19 is not divisible by 3 → not divisible by 3.
• 5: Last digit is 3 → not divisible by 5.
• 7: 5,923 ÷ 7 ≈ 846.14 → not divisible by 7.
• 11: Alternating sum: (5 + 2) − (9 + 3) = 7 − 12 = −5 → not divisible by 11.
• 13: 5,923 ÷ 13 ≈ 455.62 → not divisible by 13.
• 17: 5,923 ÷ 17 ≈ 348.41 → not divisible by 17.
• 19: 5,923 ÷ 19 ≈ 311.74 → not divisible by 19.
• 23: 5,923 ÷ 23 ≈ 257.52 → not divisible by 23.
• 29: 5,923 ÷ 29 ≈ 204.24 → not divisible by 29.
• 31: 5,923 ÷ 31 ≈ 191 → divisible by 31 exactly ✅

---

Step 2: Confirm factorization

5,923÷31=191

• 191 is a known prime number.

So:5,923=31×191

---

✅ Conclusion

5,923 is not a prime number.
It is composite, with prime factors 31 and 191.

3). My follow-up guardrails or information: “give me 5 numbers close to 5923 that are prime then”

4).  What your AI tool kept getting wrong: My AI tool (ChatGPT) falsely calculated that 5923 was a composite number, not a prime number. While calculating if it had any divisible factors other than 1 and itself, the AI divided my prime number 5923 by 31 and for whatever reason, decided to round the answer of 191.0645… to exactly 191. Which in turn, lead it to the answer of 5923 being exactly divisible by 31, deeming my original number a composite instead of a prime. In my follow up question, I asked the AI “give me 5 numbers close to 5923 that are prime then,” in which it proceeded to give me some more numbers that are actually composite.

My AI’s answer:

✅ Five prime numbers close to 5,923

5,917, 5,921, 5,923, 5,929, 5,941

The AI proceeded to give me a list of 5 “prime numbers” that are actually composite numbers (excluding 5923 itself).

5). Why do you think it kept hallucinating? I believe my AI kept hallucinating because it kept using that approximation method of calculating, and whenever it recognized that a number was very close to a whole number, it would round the decimal to an integer. This lead the AI to believe that it was a composite instead of a prime number. 

6). What are the ethical implications of generative AI hallucinations? The ethical implications of generative AI hallucinations can be pretty bad, especially for people who use AI as a search engine. If I was a mathematician and used AI to determine if a super big number with dozens of digits was a prime or not, AI might generate an incorrect response, which in turn would affect my research and so on and so forth. The effects of misinformation can ripple and the concern for it is very valid. People rarely doublecheck and/or don’t have the time to, so just seeing what the AI says is enough for most people.