When error messages attack: using ChatGPT to make peace with your users!

Cartoon: When error messages attack: using ChatGPT to make peace with your users!I’m sure everyone reading this is no stranger to the often completely unintelligible error messages that can be generated when software goes wrong!  You can even find in the most voted for ideas in the RWS Community ideas platform that is related to just that, “Get those cryptic error messages more human“.  As computers have become more and more complicated and software is  relying heavily on inbuilt technologies provided with the operating system, 3rd party libraries, and add-ons that use the APIs, it’s really no surprise that things can go wrong.  But do the messages really have to be so complicated and meaningless for the average user?  Why don’t developers make them easier to understand?

Well, three likely scenarios would be that:

  1. developers assume that the person reading the error message is a technical user who is familiar with the software and the programming language used to build it.  This means that the error message may use technical terms and jargon that can be difficult for non-technical users to understand.
  2. developers may not have considered the importance of clear and concise error messages and they may be focused more on the functionality and performance of the software and may not prioritise user experience or usability linked to error messages.
  3. developers may not have enough information about the context of the error to provide a more specific or helpful error message.  In some cases, they may not be able to predict every possible error that a user might encounter, and may provide a generic error message that doesn’t give the user much information.

Whilst I’d love to be able to understand every error message I see, I don’t blame developers at all because they are often under more pressure than we can imagine to deliver all the work they do, not only on the surface in features that we see and use, but more often under the hood having to deal with an incredible amount of change that is out of their control and yet they now have to react to.  But that doesn’t mean we let them off the hook completely!

In Trados Studio a lot has been done to try and improve things for users.  For example, take an error message like this:

Quite often all we see from the poor user experiencing it is a screenshot just like that, and often a comment saying this error has been around for years why isn’t it fixed yet!  Well that message “Object reference not set to an instance of an object” is an error that can happen when a program is trying to use something that doesn’t exist or hasn’t been created yet.  For example, imagine you’re trying to use a screwdriver, but you forgot to pick it up first.  You can’t use what you haven’t got!  Same thing with this error, it’s telling you that the program is trying to use something that isn’t there.  This can happen when a program is trying to use a piece of information that hasn’t been created or has been deleted.  It can also happen when a program is trying to use a part of a list or a group of information that doesn’t exist.  To fix this error, the programmer needs to make sure the missing information exists before the program tries to use it.  So that message in itself is simply far too generic to be helpful.

However, note the numbers, 1-4, that I added to the screenshot.  These are all things that the development team have added to try and help.

  1. if you click on this it will expand the message so you can read the stack trace (more on this in a minute)
  2. if you click on this you can save the stack trace to a folder on your computer, or your desktop, and then share it with whoever asked for it (a common question in the community)
  3. if you click on this a search will be run in the Knowledgebase based on the additional detail within the stack trace and “might” help you find a solution to the problem until it gets fixed properly
  4. if you click on this a search will be run in the Community forums for similar problems that might have been solved for the user

So this is already a good start.  But what about that “stack trace”?

The Stack Trace

… and more importantly what is it exactly?  A stack trace is a report of the function calls that led up to an error or exception in a software program. It is a diagnostic tool that provides information about the sequence of function calls that occurred before the error or exception was thrown, along with the location of each function in the code.  ok… so now we know this we probably also know we need more technical knowledge than we do have to make sense of it.  After all there’s a reason we make fun of these sort of messages… in fact apparently even developers do this on purpose sometimes to try and alleviate the frustration… not an approach we’ve taken with Trados Studio but perhaps it wouldn’t hurt!

  • “Houston, we have a null pointer!”
  • “I looked everywhere, but I couldn’t find that darn object!”
  • “Error: Your computer has developed amnesia and forgotten where it put an object.”
  • “The developers are currently investigating a missing object, please hold…”
  • 404 Object Not Found. Maybe it’s on vacation?”
  • You wouldn’t steal a null reference, would you?”
  • I’m sorry, Dave. I’m afraid I can’t find that object.”
  • Error: Object lost in space and time. Please try again later.”
  • Have you seen this object? It was last seen in memory at address x00000000.”
  • We have temporarily misplaced your object. Please stand by.”

More seriously however, what do they mean?  Well I have ChatGPT in the title of this message so let’s take one of these stack traces and ask it to explain this one!

<SDLErrorDetails time=”09/03/2022 09:06:36″>
<ErrorMessage>Object reference not set to an instance of an object.</ErrorMessage>
<Exception>
<Type>System.NullReferenceException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</Type>
<HelpLink />
<Source>Sdl.Community.ApplyTMTemplate</Source>
<HResult>-2147467261</HResult>
<StackTrace><![CDATA[ at Sdl.Community.ApplyTMTemplate.Services.ResourceManager.ExcludeWhatIsNotNeeded(Settings settings, List`1 languageResourceBundles)
at Sdl.Community.ApplyTMTemplate.Services.ResourceManager.ApplyTemplateToTms(ILanguageResourcesContainer languageResourcesContainer, List`1 translationMemories, Settings settings)
at Sdl.Community.ApplyTMTemplate.ViewModels.MainWindowViewModel.<ApplyTmTemplate>b__66_0()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.Execute()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Sdl.Community.ApplyTMTemplate.ViewModels.MainWindowViewModel.<ApplyTmTemplate>d__66.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.<>c.<ThrowAsync>b__6_0(Object state)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
at System.Windows.Threading.DispatcherOperation.InvokeImpl()
at System.Windows.Threading.DispatcherOperation.InvokeInSecurityContext(Object state)
at MS.Internal.CulturePreservingExecutionContext.CallbackWrapper(Object obj)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at MS.Internal.CulturePreservingExecutionContext.Run(CulturePreservingExecutionContext executionContext, ContextCallback callback, Object state)
at System.Windows.Threading.DispatcherOperation.Invoke()
at System.Windows.Threading.Dispatcher.ProcessQueue()
at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
at MS.Win32.UnsafeNativeMethods.DispatchMessage(MSG& msg)
at System.Windows.Threading.Dispatcher.PushFrameImpl(DispatcherFrame frame)
at System.Windows.Threading.Dispatcher.PushFrame(DispatcherFrame frame)
at System.Windows.Window.ShowHelper(Object booleanBox)
at System.Windows.Window.Show()
at System.Windows.Window.ShowDialog()
at Sdl.Community.ApplyTMTemplate.Studio.ApplyTMTemplateAction.Execute()
at Sdl.Desktop.IntegrationApi.AbstractAction.Sdl.Desktop.IntegrationApi.Extensions.Internal.IAction.Execute()
at Sdl.Desktop.IntegrationApi.Internal.Adaptors.InternalActionAdaptor.Execute()
at Sdl.Desktop.Platform.Implementation.CommandBars.SimpleActionExecuter.Execute(ActionData actionData)
at Sdl.Desktop.Platform.Implementation.ResourceLoggingInternalCommandExecuter`1.ExecuteWithResourceLogging(TCommandData commandData, String message)
at Sdl.Desktop.Platform.Implementation.CommandBars.ActionService.ExecuteAction(IAction action, ActionOrigin origin, Boolean allowToggle)
at Sdl.Desktop.Platform.WinForms.IgCommandBarAction.Execute()
at Sdl.Desktop.Platform.WinForms.IgCommandBarAction._lazyButtonTool_ToolClick(Object sender, ToolClickEventArgs e)
at Infragistics.Win.UltraWinToolbars.ToolBase.OnToolClick(ToolClickEventArgs e)
at Infragistics.Win.UltraWinToolbars.UltraToolbarsManager.OnToolClick(ToolClickEventArgs e)
at Infragistics.Win.UltraWinToolbars.UltraToolbarsManager.FireEvent(ToolbarEventIds id, EventArgs e)
at Infragistics.Win.UltraWinToolbars.ToolBase.OnToolClick()
at Infragistics.Win.UltraWinToolbars.ButtonToolUIElement.DoClickProcessing(MouseEventArgs e)
at Infragistics.Win.UltraWinToolbars.ButtonToolUIElement.OnMouseUp(MouseEventArgs e)
at Infragistics.Win.ControlUIElementBase.ProcessMouseUpHelper(Object sender, MouseEventArgs e)
at Infragistics.Win.ControlUIElementBase.ProcessMouseUp(Object sender, MouseEventArgs e)
at Infragistics.Win.Utilities.ProcessEvent(Control control, ProcessEvent eventToProcess, EventArgs e)
at Infragistics.Win.UltraWinToolbars.UltraToolbarsDockArea.OnMouseUp(MouseEventArgs e)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)]]></StackTrace>
</Exception>
<Environment>
<ProductName>Trados Studio</ProductName>
<ProductVersion>Studio16</ProductVersion>
<EntryAssemblyFileVersion>16.2.9.9198</EntryAssemblyFileVersion>
<OperatingSystem>Microsoft Windows 10 Pro</OperatingSystem>
<ServicePack>NULL</ServicePack>
<OperatingSystemLanguage>1033</OperatingSystemLanguage>
<CodePage>1252</CodePage>
<LoggedOnUser>XXX</LoggedOnUser>
<DotNetFrameWork>4.0.30319.42000</DotNetFrameWork>
<ComputerName>XXX</ComputerName>
<ConnectedToNetwork>True</ConnectedToNetwork>
<PhysicalMemory>16661104 MB</PhysicalMemory>
</Environment>
</SDLErrorDetails>

Now, I won’t post another one as it is very long, but I wanted to show how ChatGPT can help simplify this in a matter of seconds.  I just asked “Can you explain the likely cause of this error?” and pasted the full error stack into the chat.  The result:

Not only is it shorter, but the explanation is something anyone would be happy to read, and could even see it looks as though the problem is caused by the applyTMTemplate app from the AppStore.  So removing the plugin would probably remove the error and at least you could carry on working.

But if that’s not enough I can refine this answer..

Even better!

Now I’m pretty sure that you won’t get this level of certainty with every error message, but what you can achieve is this:

  • an understanding of the problem: with a simplified explanation, you can get a basic understanding of the problem and why it occurred.  This can help you to better communicate the issue to a developer or support team and ask more informed questions
  • troubleshooting: with a basic understanding of the problem, you may be able to take some basic troubleshooting steps on your own. For example, you may be able to check the input parameters to the method in question or perform other basic checks to help narrow down the cause of the problem such as splitting a file and testing each half, repeat and try again.
  • awareness: even if you are not a developer, having a basic understanding of common errors like NullReferenceException can help you to be more aware of potential issues and avoid making mistakes that could lead to such errors.  For example, if you are specifying a rule to extract the value of an XML attribute, you should check that the attribute is present before trying to extract its value.  If the attribute is not present, trying to extract its value could result in a NullReferenceException.

It’s not always going to be something you as a user can help, and I’m not suggesting for one minute that you should have to.  After all you paid for the software and it should be perfect… shouldn’t it?  But I think having a better understanding of the error message and what it really means could really help.  If nothing else perhaps you’ll have a better understanding of why that “Object reference not set to an instance of an object” is still there after a decade of providing the software!

Conclusion…

To wrap up I asked ChatGPT this:

Will software always have errors in it or can the perfect error free application be created?

The response:

It is unlikely that a perfect, error-free software application can ever be created. The reason for this is that software is created by humans, who are fallible and make mistakes. Even with the use of rigorous development processes, thorough testing, and careful quality control, there is always the possibility of errors and bugs in software.

Furthermore, software is often designed to work with a variety of hardware, operating systems, and other software, all of which can vary in their configuration and behavior. This can create unforeseen interactions and compatibility issues that can lead to errors.

That being said, software development practices have improved significantly over the years, and there are many tools and techniques available to help minimize the likelihood of errors in software. These include automated testing, code reviews, and quality assurance processes. With careful attention to these factors, software can be made much more reliable and robust, even if it can never be completely error-free.

Well… I can’t leave that there!!  So I ask:

Will AI be able to achieve what humans have not and create the perfect error free application?

The response:

While AI has the potential to improve the reliability and quality of software, it is unlikely that it will be able to create a perfect error-free application. This is because creating software that is completely free of errors and bugs is an incredibly difficult task that requires human judgement, experience, and creativity.

AI can certainly help in the development process by automating testing, identifying potential issues and suggesting improvements. However, AI is only as good as the data and algorithms it is trained on, and it may not be able to identify all possible errors or account for all possible edge cases.

Furthermore, even if an AI system could identify all possible errors in a piece of software, it may not be able to fix them without human intervention. Fixing errors in software often requires careful analysis and understanding of the underlying code, as well as creativity and problem-solving skills that may be difficult to replicate in an AI system.

In short, while AI can certainly improve the reliability and quality of software, it is unlikely that it will be able to create a perfect error-free application on its own. It will require the input and guidance of human developers to achieve this goal.

A good, and realistic answer… for now!  What do you think?

One thing for sure, if you’re not using a tool like ChatGPT already I think you’re missing a trick.  Most people don’t want to be helpless and when you can’t understand an error message that surfaces just when you’re under pressure to deliver that project the frustration is only going to hurt you.  Now you can not only report the problem more effectively but you might even be able to solve it yourself!

8 thoughts on “When error messages attack: using ChatGPT to make peace with your users!

  1. This really sucks, I am baffled. Love this explanation, as even being quite experienced user I would never be able to get so detailed conclusion from the original description.

    1. Sucks because there is technology that can do so easily a task that would be difficult for us without considerable technical knowledge? Or sucks because you don’t like the article?

  2. This particular error message is what I receive most often. The cryptic error messages does not really disturb me, but processing stops, rather than continuing with the next translation segment (which often would be possible) and often there is no reference to the associated “faulty” segment.
    Indeed, often I cannot even continue working until the “cause” is corrected. In my case, by “trial and error”, I found that the problem is with terminology recognition that at least can be “solved” by temporarily disabling verification. Perhaps chatGPT could help here? But this raises the question of where chatGPT gets its “knowledge”; with my few queries, it seems to access only a limited Internet subset.

    1. I think the point is this particular error message has probably been “fixed” many times! Often the only way to get to the bottom of fixing a particular reason for this is to be able to reproduce it in a development environment. The error stack can often provide the temporary solution, and can certainly help a trained eye to see where the problem comes from, but you can never be sure you fixed it unless you can reproduce it, fix it and then fail to reproduce it again. This is why we always like to have more information.

      But this raises the question of where chatGPT gets its “knowledge”; with my few queries, it seems to access only a limited Internet subset.

      If you ask it the answer is something like this…

      “It was trained on a large dataset of text from various sources, including books, websites, articles, and other documents. Its training data comes from a diverse range of topics and fields, allowing it to have knowledge on many different subjects. Its training dataset covers a period up until 2021 which means that its knowledge and information are based on data available up until that time, and it doesn’t have access to any information or events that have occurred after that date. However, as an AI language model, it’s designed to understand and interpret natural language, which allows it to have conversations on a variety of topics, even if it doesn’t have access to the most up-to-date information.”

      So for some things the answers can be very wrong even if they sound right!
      When it comes to more technical stuff…

      “As an AI language model, it can provide some guidance on how to approach computer code-related questions and error stacks, but it cannot directly evaluate or execute code. When it comes to code-related questions, it can help with:

      • Syntax and language-related queries: it can help clarify the syntax or the meaning of a particular line or function in the code.
      • Algorithm and logic-related queries: it can provide insights into the logic or algorithm used in the code, suggest alternative approaches, and discuss the pros and cons of each approach.
      • General coding best practices: it can share best practices related to coding style, performance optimization, and error handling.

      When it comes to error stacks, it can help with:

      • Understanding the error message: it can explain what the error message means, what caused it, and how to resolve it.
      • Suggesting possible solutions: based on the error message and code context, it can suggest possible solutions or approaches to resolve the error.

      In summary, while it can’t execute or evaluate code directly, it can provide guidance on syntax, algorithm, best practices, and error messages to help users better understand their code and resolve any issues they may encounter.”

  3. I tried ChatGPT on one of my error messages, but it seemed only to process the first part. What did you do to process the complete message?

    1. Perhaps you’re using the free version which is subject to heavy restrictions over usage. I use ChatGPT Plus so not only get a better choice of models but also have some guarantee of performance.

Leave a Reply to Anthony RuddCancel reply