All that glitters is not gold…

001Years ago, when I was still in the Army, there was a saying that we used to live by for routine inspections.  “If it looks right, it is right”… or perhaps more fittingly “bullshit baffles brains”.  These were really all about making sure that you knew what had to be addressed in order to satisfy an often trivial inspection, and to a large extent this approach worked as long as nobody dug a little deeper to get at the truth.  This approach is not limited to the Army however, and today it’s easy to create a polished website, make statements with plenty of smiling users, offer something for free and then share it all over social media.  But what is different today is that there is potential to reach tens of thousands of people and not all of them will dig a little deeper… so the potential for reward is high, and the potential for disappointment is similarly high.

My guess is you can probably relate to this in some form or another, but as I like to write about translation technology I wanted to use this to look at a new breed of CAT tool that claims to support around 30 filetypes (Studio supports over 70 plus any available through the API/AppStore) including *.SDLXLIFF, and also has the ability to support the Studio translation package, *.SDLPPX.  Now the title of this article is “All that glitters is not gold”, and I called it this because this particular tool, SmartCAT, is free and if you read their rhetoric they can provide everything every other major translation vendor can.  They make their revenue by taking 10% commission on translation services ordered through their site, and also for machine translation if you use it (Google, Microsoft and SmartCAT) and they provide a free online translation environment to support the services.  Sounds perfect, and this is backed up by their often used quote”everyone wins, translators, customers and us”.  I like the model… what I don’t like are some of the claims for it’s technology.  The reason I don’t like them is because it has the potential to cause a lot of problems for translators and their customers when it doesn’t do what it says in the glossy website.  Perhaps I should just ignore their inappropriate commenting of their solution in forums and social media… perhaps not as I think these things shouldn’t go unanswered.  But aside from the glittery claims, the solution is also online only and for me this is a non-starter.  Perhaps it’s my age… or perhaps it’s the frustrating hours I have spent travelling with an inability to get an internet connection that prevented me from doing any work that was not possible locally.  Online is certainly the future, but I think we are a long way off being able to take this step without any sensible alternative.

Digging a little deeper…

I’m going to focus on the supply chain workflow using *.SDLXLIFF files and *.SDLPPX packages as these are the areas I am most bothered about.  The ability of this tool to handle other filetypes is something it’s quite easy to see… it can open the filetypes it mentions but as a professional translator have a think about the options you often need to use when preparing files.  With SmartCAT there are no options (other than to tag things using regex afterwards), it simply parses all the text.  This might be passable for a part time translator looking for a simple way to handle a one off job, or where all the text being parsed is translatable, but it may not be adequate for a professional.  If you look at XML file handling for example there is nothing smart about it at all, everything is parsed if it’s in an element; no ability to handle anything other than an element and no ability to pick and choose which elements.  Even something simpler such as a DOCX does nothing more than parse the simple stuff… no field values, no tracked changes etc.  I’m sure for the simplest of files this tool might be fine, but I really doubt it’s for a professional translator who needs more flexibility than this.  This is my problem… Studio, Trados, memoQ, DVX… all tools designed for a professional translator.  SmartCAT is not in the same league, but from reading the website you could be forgiven for falling for the glitter, especially if you were new to the industry.

The supply chain workflow

When you’re working with source files direct for the end client then you take responsibility for the final job.  You have to deal with missing translations, or poorly formatted files.  But when you are dealing with translation packages or bilingual files you are most likely handing them back to a project manager who has to deal with them.  This project manager will have spent time preparing the files so that they are handled in the way they choose and they expect to see the files back fully translated and in the same format they sent them.  Anything less is going to create a lot of work, and potentially risk non-payment for the translator.  So now, if you don’t handle the files and packages in a supporting tool you are affecting more than just yourself.  But let’s take a look at a few examples based on some very simple testing.

Lack of support for languages

SmartCAT claims support for 70 languages (including variants); Studio supports nearly 100 variants of English alone… the total number of languages supported getting onto 600.  This is a limitation… fair enough.  If you try to add a package containing an unsupported language you get this message:

002

That’s also fair enough.  It does limit its ability to be able to “support” Studio packages but at least you are told at this point even if there is no mention of it in the glitter.  If you just load an SDLXLIFF however the story is different and you can change the languages to be whatever you like… both the source or target.  There is an argument to say this is ok, in fact its great.  But for the project manager who sends out *.SDLXLIFF files only this may not be so great.  In this case SmartCAT overwrites the language code with the one you select and the file you send back to your project manager is no longer correct.  Not so smart.

Perhaps worth mentioning at this point that if I load a partially translated SDLXLIFF into SmartCAT then it presents me with a completely empty target column, so all the work carried out by others is lost.  Not so smart support for SDLXLIFF.

Segment statuses and comments

Project Managers use comments to send information to the translator, and get comments back.  Project managers use different statuses for the segments to differentiate between the work required and to help identify which segments need to be looked at, and which don’t.  SmartCAT is a little like the infamous honey badger in this respect; he doesn’t give a f…hoot!

003

I aligned the Studio statuses on the right with the SmartCAT interpretation on the left.  In general SmartCAT considers “translated”, “reviewed” and “signed-off” all as “done”.  It considers “draft” and “not translated” where source is copied to target as “in translation”.  This all provides no way for the translator to know what to work on at all and to make matters worse when you save the target from SmartCAT every single segment that is “done” now opens in Studio as “signed-off”.

As for the comments… what comments?  Definitely not very smart.

Support for translatable controls

What do I mean by this… basically anything that uses a non-translatable feature to protect parts of the text.  This is a very common requirement in localisation projects and SmartCAT doesn’t handle this at all well.  One quick example creating a Studio project and then opening the package in SmartCAT:

004

I have no idea what’s going on here but clearly the support for packages and sdlxliffs is simply not there.

Conclusion

I’m going to stop looking at this here because I think I’ve made my point.  I haven’t even written about what SmartCAT does to the return files and I could easily write a lot more about this; nor have I looked at how SmartCAT is unable to handle other features in Studio and show how they are represented in the packages/sdlxliff files; nor have I looked at their claims to be able to support MultiTerm XML… at least I have not written in here but I have looked.  There is a lot I haven’t looked at, and on this basis alone it appears SmartCAT have not looked very well either.

I think the points I really want to make are these.  First, don’t believe everything that you read; the glossy websites and tweets telling you that a tool like this is a suitable substitute for your work in Studio are simply not true.  I imagine the same thing would apply to the features in other specialist translation tools as well.  Secondly, take the trials and make your own tests if you want to be sure.  Thirdly, if you’re thinking about using SmartCAT to support Studio projects and want some impartial advice (remember what I said about not believing everything you read, although I will be honest) then feel free to contact me in the comments.

Finally, I don’t want this article to sound completely as if I’m having a go at SmartCAT.  I am frustrated by their constant inappropriate marketing messaging, but that doesn’t mean they are all bad.  They do have some nice features and in an environment created around files they can work with I can imagine this is a very nice solution.  But it’s not all things to all men, and certainly not gold!

0 thoughts on “All that glitters is not gold…

  1. Thanks Paul, I also agree with you. I did a fast check on their website, sounded too good to my taste. Well the results were very poor and it was clearly identified as “high risks”.
    I did also some free testing and I agree with you, this is a joke.
    I get a massive amount of mails from these guys so bad in fact that I reported them to the French Internet Police (yeah I know, sounds stupid, but we have laws in France and when you tell them that you do not want their mail anymore, it is a felony to continue with the spamming). Well the guy I spoke to, knew those guys quite well and I cannot mention here what he told me, but it was not very funny.
    This is the Uber version of translation but with a headquarter in Russia ?
    I do not like this at all.
    Best regards,
    Ivan

    1. Sorry to hear we were bugging you with these emails!

      Actually you could unsubscribe at any time by clicking “Unsubscribe” (would work quicker than reporting to the Internet Police I guess:). Sorry if this wasn’t made clear enough in the emails.

      Anyway, we’ve unsubscribed you now, and we won’t be annoying you anymore 😉

    2. By the way, according to your public profile on SmartCAT, you have translated 18 (eighteen) words on the platform. Is this the testing that led you to the above-mentioned conclusion that “this is a joke”? If yes, we’re impressed by the speed and efficiency of your reasoning!

      We’re also happy that you did fill in your profile and thus, we assume, are willing to take on jobs from SmartCAT customers. We’re always excited to see translators with such an extensive experience in our marketplace.

      Happy New Year, and good luck with all your endeavors in 2017!

      1. I did some extensive testing of your tools, but on a different email. Yes I know, this is not completely fair, but this is life as well.
        You seem to be very sensitive to any kind of remarks, this is most interesting.
        Have a nice 2017 too.

  2. Paul, there are a couple of things that don’t quite sit right with me with this article and your recent argument with SmartCAT during the Christmas break (!) on Twitter. You are right to point out that the claim alone to support a certain file format does not mean much — users have to really test it to make sure that it works for their purposes — but that’s been written about a bit in the past and I trust that professionals understand that and do the appropriate testing. But then, that’s certainly true for Trados in the past as well and even the present do some degree (try to compare the processing of image-based PDFs in Trados vs. SmartCAT, something that Trados claims it can do and does, unlike SmartCAT, not do particularly well — or of course just images which Trados does of course neither claim nor support — or look at the dictionary integration that SmartCAT has in some languages or the smart way of safely working with several team members simultaneously in one file). As you know, I have no more allegiance to SmartCAT or its makers than to Trados. I also know that Trados is a more mature tool but just like any tool it has its weaknesses as well. So, why do I mention all of this? Because I think that we (you?) need to relax a little and enjoy the holidays.

    Happy 2017.

    Jost

    1. Jost, thank you for your comment. All tools have their good points as I did mention, but interesting to see you coming to their defence like this though. Even more interesting that you don’t appear to accept that it’s not ok to offer a solution that is damaging to a translation workflow by mishandling of the intermediary files. I don’t have a problem with SmartCAT, I don’t even have a problem with them trying to handle the SDL intermediary files and packages, but I do have a problem with their reckless marketing approach. Perhaps if you were on the receiving end you might think differently.
      Happy 2017.
      Paul

  3. Thanks a lot for your article, Paul!

    Our engineers are already on your issue reports. Some of them were already known and will be fixed as early as in the next release (due in several weeks), and we’ll definitely do our best to wrestle the rest.

    To fully support a format as complex as SDLXLIFF takes time — especially when the original vendor does not disclose its specifications (and I’m sure it has good reasons not to;-). But given the number of requests we get from customers who want to switch to us from that original vendor, we’ll work hard to make this happen sooner.

    I also fully agree with your suggestion to “take the trials and make one’s own tests.” A good thing about SmartCAT is that testing it is the same as using it. So a user won’t have to pay half a grand to get unexpected bugs that will mess up its translation workflow (something we will revisit a bit later).

    Once again, thanks for posting this — it’s inspiring and motivating that SDL considers us blog-worthy.

    Here’s to an interesting 2017!

    PS. Doesn’t gold, well, glitter?

    1. Thank you for posting, and for confirming you are working on the fixes for these few issues. I’m looking forward to seeing whether you fix any more. SDLXLIFF is basically a valid XLIFF with a few custom extensions and the API documentation we provide, freely available to anyone, makes it clear how to parse one using the API.

      Worth noting that this is a personal blog and not SDL, but to see you responding to me like this is similarly inspiring and motivating. Whether SDL think about you at all is an interesting question…

      1. We believe “a few custom extensions” is a bit of an understatement. Thomas Vackier of Yamagata Europe once published a useful article titled “The double-edged sword of extensibility in XLIFF” — still as relevant today as it was 6 years ago: https://www.localisation.ie/oldwebsite/xliff/resources/presentations/xliff-vackier.pdf

        Also, as useful as API documentation can be, we don’t think it is the same as file format specification.

        1. Indeed this article is very relevant. But I think you’ll be hard pressed to find anything in the sdlxliff that uses custom extensions where there was something available in the standard xliff to support it. All you need to do is at least preserve anything you are unable to handle, but you are not doing that, at least not at the moment and this is the problem I have with your claims. Perhaps a better approach for you is to support a different intermediary like TTX perhaps? There is a free tool available for all licenced Studio users that can convert SDLXLIFF to TTX and back again, without any problems. That would work until you figured out how to do what other developers have already managed without a specification.

          Incidentally, I was interested to look at your API documentation but you don’t seem to publish it at all? Is it freely available?

Leave a Reply to SmartCAT (@SmartCATai)Cancel reply