Recent searches
Search options
Oh yes, GPT-4 will happily tell you to shift the keyboard focus to the form field which has an error and fire off an alert message, even though those two things can be an anti-pattern: The focus shift will most likely make the alert not surface. It took a bit of prodding to get it to suggest instead to use a link that takes the person to the error field next to the message, because it still leaves user control. This is why I worry for AI used for accessibility work by non-experts.
It simply can't really infer based on your context the exact accessibility needs and requirements for the code. It can generalize, and yes, shifting focus can sometimes be good for a form field, like when you open a new dialog, or just a user action triggered it to happen for the specific scenario of the page. Even for error management it can work, if you aren't alerting and it's a simple form, but it fails to get the nuance of what those two things as an experience create.
It also matters how you give it the instruction in tonality. If you tell it, "I think I did it right! This focus management works great!" or give it a tree-grid that is marked up right but missing some arrow key for expand/collapse, and say "I wrote this super well, didn't I?" it will take your side more often rather than be critical like, "uh, no, you actually missed some keyboard interaction here." But if you say "I think I'm doing this wrong", it will point them out more likely than not.
@Tamasg I think/hope people go through a process where they ask it something and the response seems mostly right at first. Then they have it opine on something they are an expert in, and it gets everything wrong. then they realize that it's just repeating what it heard everywhere else and is not very authoritative, and that informs their understanding of the rest of the output. Much like cable news!