My AI chatbot rolled out a buggy update and started giving customers wrong refund amounts.

I was at my desk in Austin last Tuesday when I saw 3 support tickets about people getting $500 instead of $50 refunds, so I had to manually revert the model and reimburse the difference out of pocket. Has anyone else had a production AI go rogue on a simple math task without warning?

1 comments

1 Comment

ruby_lane1d ago

Oh man, that's rough. I saw something similar with a photo tagging AI that started adding faces to completely blank backgrounds. The refund thing is scary though because it's actual money leaving the building. Three tickets in one day sounds like you caught it fast, but still a nightmare to undo. Hope the math models get better at, you know, math.