AI and Community

I’ve written quite a few articles in the last year or so on the use of AI in a localization setting, and in general as a tool to help you complete technical tasks you may not have been able to do without help until now.  Certainly I’ve been making extensive use of this technology to simplify my life in many ways since I can now achieve in a few minutes things I could only wish I had time to learn until now.  But I thought I’d share a little of the practical use of AI within a community setting as we’ve made great use of this technology to help our users in the RWS Community and I think it largely goes unnoticed.

So I’m going to cover the following topics where we have used it, but just at a high level:

Forum Finder

The name says it all… if helps you to find the forum you should be using.  This is important because you’re more likely to get the right help if you post your questions into the right place, and you make it easier for other users and also the forum administrators because topics will be properly grouped together without making admins do the work of moving posts into the right place because you were unable to see where to post.  Is it that hard… well some users tell me it is and that is really all I’m going to say!

How does the AI solution work?  Well let’s say I ask this question, “My licence key for Trados Studio isn’t working, please help.”… a fairly typical sort of question.  If I ask this in the Community Feedback forum (which let’s face it, is much harder to find!!), and many people have done this in the past, then Trados AI kicks in and suggests a better place before you post:

Too easy… how about “I need help with an AHK script to move selected text in Trados Studio.” posted into the Trados Studio forum?

I could go on, and this works across the community using a trained TradosAI model.  Before we had this solution we used to move at least a dozen posts every day.  That in itself is a good reason to provide a solution like this, but more importantly users posting in the wrong place won’t waste time emailing everyone they know in RWS because they haven’t had a response after posting somewhere that doesn’t get as much attention, or the moderator simply didn’t know where to move it to without asking half a dozen people in the company first!!  Now however we hardly move any as everyone pretty much gets it right the first time!

Trados AI

Once you’ve got your post in the right forum how about a little help to answer your question and not have to wait for a human?  You may have noticed that when you post a new question you see this underneath it once it appears in the forum:

To demonstrate how effective this can be, and how helpful it can be I created a post in the RWS Community where you can see the post and the TradosAI response:

“Can you help me with a small problem I’m having creating a custom XML filetype in Trados Studio 2022?”

The process is very straightforward and you can read the detail of how to work with it in this Community wiki.  But essentially this is all you need to know:

1. Click on the “Don’t wait for a human, try TradosAI!” link
2. The answer comes as a verified answer.  If you don’t like it click on “Reject Answer” and provide more information and Trados AI will have another go.
3. Note that this is just a machine!!  So don’t expect it to be a mind reader!  Often the Trados experts in the community are mind readers and can help even when you may not know how to ask about the topic you need help with.  TradosAI doesn’t always have that kind of intuition even if it can be pretty good; so always provide as much information as you can and explain your problem as clearly as you can.

Do all that and it can be a really cool tool.  It’s basically OpenAI with some assistance from Google and training from many sources such as the Trados help documentation, blog articles (mine and select others in the public domain), verified questions and answers in the RWS Community (so be a good citizen and mark correct help when you get it as verified since this all helps to make TradosAI better), the RWS Knowledgebase, the really old MultiTerm manuals (I love that bit… we got the PDF versions of the old ones pre-2007) as they are still the best resource for MultiTerm questions around and more!

We also provide additional help for moderators of the posts who can use the same TradosAI to get an answer to a question, but they can can also “tidy it up” as needed, or completely ignore it if they don’t like it.  This can be really helpful, especially with error messages which can be deciphered manually if you know what to look for… but it takes a hell of a lot longer!!

It’s called the “Moderator Assistant” and just to give you some idea of how that works, it looks like this:

So this gives a moderator the ability to reject the answer altogether and just write their own, or accept (with moderations if they choose) the AI generated response.

TradosAI isn’t always right!  I’m sure that most users will know by now that hallucination is a problem, so it’s always a good idea not take things at face value.  Always test the response, if it’s right than you’re onto a winner!  If it’s wrong then explain why and try again… often this can help improve the answer.  In fact asking poor questions with insufficient context is quite likely to generate a hallucinogenic response.  So take the time to ask a good question with appropriate information will nearly always yield a better response.

This of course leads into the question of how do we prevent hallucination without reducing the effectiveness of it being able to solve questions it has not been specifically trained on… such as the very question I asked here?  Well this we do with two things.. mainly:

1. a well designed prompt built into our code
2. an editable database that we use for the training

I’m not going to address the prompt, but I will show you the basic idea of this website and database:

There are more features that we added to this but the basic idea is that when we see a poor response we can search for where it may have come from and make a correction to the data that is used to fine tune the model.  We can also load Excel/CSV files of Q&A that has been verified from whatever sources we took the data from, so this give us pretty good control at a non-developer level to be able to have a team of contributors who can work with us to continually improve the data that TradosAI is able to work with.

We have taken this concept for use elsewhere in our business, but I love that it was our community team that did this independently of our development organisation because this is just one of the things I have always loved about working in this company… if you have some good ideas then you will always have the freedom to run with them and innovate if you want to.  For me this is a priceless feature in building great teams, being successful, and keeping happy employees!

TradosAI Generated Image Alt-Text

You may have noticed that RWS are on a “journey to inclusivity!”.  This is a topic I’m quite invested in, so I was really pleased to be able to use AI with the RWS Community team to solve a problem in the community with the digital accessibility of images.  Now if you’re not sure what I mean there’s a short and accurate explanation from the European Blind Union here.

Within the RWS Community there are tens of thousands of images that users share to help explain their questions, or provide a response.  I feel on safe ground saying that the vast majority of users don’t add the alt-text to their images when they post them.  I didn’t either as I only started to do this diligently myself during the last 12-months or so.  It’s not that difficult to do, but it does take time and if you copy paste the image in instead of selecting it and loading it then you would also need to edit the image that was already there… something nobody will ever do unless they are passionate and diligent enough to make the effort.  If you don’t know what I mean it would be adding explanatory text to here:

The “Alternate Text” field is read by a screen readers such as JAWS (Job Access With Speech) or NVDA (Non Visual Desktop Access) and these tools can provide feedback using synthetic speech and Braille so that a blind user knows what the image is about.

Can you imagine trying to use the RWS Community with so many images that are used instead of, or to complete, the description of a problem or a solution if you cannot see the screen at all?  Well this is the problem we set about using AI to solve.  We actually created two solutions:

1. We automatically create the alt-text for every image that goes into the forums in the background and add it to the image
2. We can also select a start and end date and run this solution over a historical period to make up for the time we didn’t have this solution in place

In fact, we actually ran this back to the beginning of 2018 and added the alt-text to around sixteen thousand images… it took nearly 2-days to complete!  The solution itself doesn’t just use the image as this often doesn’t explain enough, or it shows too much information without context.  So we also use the content in the thread to try and make sense of it and provide a more reliable alt-text.  The solutions are actually pretty good!  You can go and check these for yourself as you’ll see in existing posts, or when you add an image, then some text like this will appear at the bottom of your post if there is an image in it:

Generated Image Alt-Text
[edited by: Trados AI at 2:47 AM (GMT 1) on 31 Mar 2024]

In fact you can check that one I just referred to in this post as I didn’t post it so it probably carries more weight!  Checking it is easy, just right-click and inspect the image… you should get something like this:

The generated alt-text is pretty accurate!!

alt=”Screenshot of Trados Studio interface showing the Editor with a translation project open. The Termbase Viewer pane is displayed with a small font size for the term list, while the selected term ‘Petitioner’ in English and ‘Ricorrente’ in Italian is displayed in a larger font size.”

What do you think?  The text within the question posted is what makes this as accurate and helpful as it is.

The only drawback to this solution so far is that the RWS Community sits on a product called “Verint Community” which not accessible.  So even though we can do whatever we can to make these posts accessible, until Verint see this as important as we do and make the necessary changes for their platform to be accessible it’s going to be challenging for a blind user to work with the community itself.  They do know what’s needed… we just need them to do it!  It’s a great platform and we heavily customise it via the APIs… but we can’t do everything without a little more help!

I’m really pleased to see the enhancements to some of the standards and directives we use in Europe are tightening up so that the old and rather loose definitions of what was needed to be accessible are being updated with requirements to actually be usable by blind users.  It will make a difference… here’s two in particular:

• European Accessibility Act (EAA) – directive comes into force in June 2025
• Web Content Accessibility Guidelines (WCAG) 2.2 – guidance released in October 2023

This space is one area where AI is already proving to be a valuable asset in improving the lives of blind users, so this particular use in the RWS Community is fantastic to see.

Sentiment Analysis

This one is as much fun as it is serious… we use AI to determine the mood of two things:

1. Individual posts into the forums
2. Complete threads in the forums

You can find a wiki in the RWS Community that explains how the first one works and why we do it so I won’t say a lot more other than to share the main effect for you and that is that every post you make will get a score and an emoticon similar to these:

In general when you’re pretty unhappy about something and reflect this in your post then we’ll immediately see this just by scanning the posts.  Similarly if you’re really happy, maybe you received a good response that helped you, then we should be able to see that too.  This is useful for several reasons:

1. It’s good to draw your sentiment to the attention of our moderators on a post by post basis.
2. We can also track posts by sentiment and even identify users who are consistently unhappy which gives us an opportunity to engage appropriately if needed… we don’t want unhappy users!
3. It’s good for us moderators to see what sentiment our responses could have too… and in our case help us rewrite something if we feel the need.  I am personally guilty of replying only to see that the sentiment was poor when I didn’t mean it the way it might be taken… especially by a non-native English speaker.  So if noted I can correct it.
4. And it’s multilingual 😉

They were the main reasons we did this, and that leads me onto the second reason we use AI for sentiment and this would be to measure “Complete threads in the forums”.  You won’t be able to see these, but as moderators we can see the sentiment of the overall threads like these for example:

I tried to find some examples of each type of sentiment we consider and these can be seen by the different emoticons used in the images above.  The concept behind the emoticons is to visually represent the overall sentiment of a forum thread.  By plotting the sentiment score of each post on an xy graph, we can draw a trendline that reflects the collective mood of the conversation.  The direction and steepness of the line – its gradient – symbolise the general feeling within the thread: an upward slope suggests positivity, while a downward trend indicates negative sentiment… for example, five posts in a positive thread where the user may have been unhappy to start with, but as the thread went on the understanding and responses became more and more positive (I’ll let your imagination think of the graphs for other results):

However, the sentiment isn’t solely based on the line’s direction; it also considers the sentiment scores themselves.  So if the average sentiment score remains above 3 (on a scale where higher numbers are more positive), the thread may still be considered positive even if the trendline is slightly negative, acknowledging that a high starting point can set expectations for some decline.  The reverse is true for an average below 3.  By integrating both the gradient and the actual sentiment scores, a nuanced overview can be achieved, leading to the assignment of an emoticon that best reflects the thread’s overall tone – from positive to neutral to negative.  This method provides moderators with a dynamic and contextual analysis of forum threads, providing a quick visual cue of the community’s experience and feelings throughout the lifecycle of a thread.  This also means we can draw attention to specific threads where necessary and see whether we can do anything to address the concerns raised.  Not always possible of course… but when we look at this in our overall reporting dashboards it’s helpful to get a balanced view of the community and the experience of our users… probably better than running surveys where only a very tiny proportion of our userbase ever respond!

What conclusions can I draw from this?

We started using AI in the community because like everyone who started to think about the different ways in which we could use AI when it hit the headlines early last year we wanted to see how this type of technology could be used to help in our part of the business.  Having got this far I think the use of AI in community settings provides a good example of how technology and human interaction can coexist to create a superior customer experience.

AI is not positioned to replace human moderators and community managers but to complement them.  By taking over repetitive or straightforward tasks, AI really can free up human resources to focus on more complex issues that require emotional intelligence, empathy, and a deeper understanding of context – qualities that AI cannot replicate yet!

With AI handling common queries and routing, users can benefit from instant support.  This hopefully will lead to increased satisfaction as the the AI improves and users get their questions answered quickly.  Human moderators can step in to handle the more nuanced conversations or follow up on initial AI interactions that were clearly smoking something!

The sentiment analysis provided by AI can reveal patterns and trends in user sentiment that might go unnoticed by humans alone.  Community managers definitely have a feel for this as they are in there every day, but quantifying it can be difficult.  These sort of insights can be used to understand broader user issues and concerns, and in turn help to drive more strategic improvements to the community and product offerings.

The AI-generated alt-text for images is an example of how AI can directly contribute to a more inclusive environment.  By making sure that all community content is accessible, AI ensures that no user is left behind, reflecting a brand that cares about all its customers… once Verint sort the more general navigation issues out!!

AI and humans working in tandem creates a loop of continuous improvement… not just hype, it really does!  AI learns from the interactions curated by humans, and humans adapt strategies based on AI-generated data, leading to ongoing enhancement of user experience.  Before we started doing this I don’t think we really paid as much attention to the detail at this level so this is definitely a great learning experience for everyone.

From a  business perspective, which does have a positive knock on effect to customers, AI allows communities to scale their support functions without a proportional increase in human resources which might put prices up.  As the community grows, AI can handle the increased volume of routine interactions, while humans can focus on ensuring the quality of engagement remains as high as possible.

In essence, AI acts as a force multiplier for human effort in community settings.  By handling routine tasks and providing analytical insights, AI allows humans to concentrate on creating meaningful relationships, providing empathetic support, and strategically developing the community based on insights gleaned from AI data.  This synergy can lead to a community where both users and owners feel more valued and understood, fostering a sense of belonging and satisfaction… at least this is where we want to be!