Archive

Tag Archives: filetypes

I’m back on the topic of PDF support!  I have written about this a few times in the past with “I thought Studio could handle a PDF?” and “Handling PDFs… is there a best way?“, and this could give people the impression I’m a fan of translating PDF files.  But I’m not!  If I was asked to handle PDF files for translation I’d do everything I could to get hold of the original source file that was used to create the PDF because this is always going to be a better solution.  But the reality of life for many translators is that getting the original source file is not always an option.  I was fortunate enough to be able to attend the FIT Conference in Brisbane a few weeks ago and I was surprised at how many freelance translators and agencies I met dealt with large volumes of PDF files from all over the world, often coming from hospitals where the content was a mixture of typed and handwritten material, and almost always on a 24-hr turnaround.  The process of dealing with these files is really tricky and normally involves using Optical Character Recognition (OCR) software such as Abbyy Finereader to get the content into Microsoft Word and then a tidy up exercise in Word.  All of this takes so long it’s sometimes easier to just recreate the files in Word and translate them as you go!  Translate in Word…sacrilege to my ears!  But this is reality and looking at some of the examples of files I was given there are times when I think I’d even recommend working that way!

But there were files I saw that looked as though they should be possible to handle in a proper translation environment.  We tested a few and the results were more often than not pretty poor.  So even though we could open them up it was still better to take the DOCX that Studio creates when you open a PDF and then tidy up the Word file for translation.  At least this is some progress… now we’re able to handle the content in a translation environment and not have to recreate the entire file.  But it would be even better if the OCR software could make a better job of it.  And this is where I want to get to… better OCR!

SDL Trados Studio 2017 continued to provide the same PDF filetype that uses technology from SolidDocuments in earlier versions of Studio, and this does a fairly good job of extracting the translatable text with OCR for many files.  But it could use improvement.  SDL Trados Studio 2017 SR1 has introduced another option for OCR using a software called ReadIris that is part of the Canon Group.

Out of the box, according to the documentation, Iris supports 134 languages for OCR which is pretty impressive.  They don’t quite match the languages supported by Studio however, but a rough count and compare suggests there are some 95 shared languages… and they even support Haitian Creole which Studio does not as we know 😉  Still impressive however and it easily beats the 14 languages supported by Solid Documents in Studio 2017 prior to the introduction of Iris.  Additionally this opens the possibilities for handling scanned PDF files in Asian languages, Arabic, Hebrew and many others that were previously difficult, if not impossible, to handle.

Using the new options

So let’s take a look at where you can find this new option and how you use it.  First of all you need to go to your options:

File -> Options -> File Types -> PDF

Then navigate down to “Converter“.  Down near the bottom you’ll see the “Recognize PDF text” group as shown below and the option to activate this new feature is at the end:

Check the box and you’ll be presented with this screen:

It’s an App!  You may be wondering why you need to do this and why it was not just integrated into Studio?  The reason is simple… not everyone will want this option and the underlying software requires a 150Mb download which would have increased the size of the Studio installer to over half a gigabyte.  So it was made optional.  If you want it you click on the “Visit AppStore” link in the message above, or the one I just wrote, and download and install the plugin just as you would any plugin from the appstore.  If you don’t do this then Studio won’t be using the software.  There are no warnings, and the option remains checked, but you won’t be using it.  So when I open the Chinese PDF I just created by copying some text as an image and saving it to a PDF all I’ll get is this:

None of the text is extracted for translation at all.  But if I install the plugin and try again I see this:

Now we’re cooking!  Would be useful to get rid of the tags though as these seem to be aesthetic only, just colours and font changes where the OCR picked up a few minor differences and then introduced tags to control them.  As these are formatting tags only I could just ignore then, or press Ctrl+Shift+H to hide them in the editor.  But if I want to remove them altogether I can do this with another app. called Cleanup Tasks that I have written about before.  These three options do the job for this file:

Now I have this and can translate without any tags at all:

Nice… and if all of that sounds complicated it wasn’t really.  I created a short…ish video below putting this all together so you have an idea of how it works.

Approx. length : 16.26 mins

After all of that I don’t want you to get the impression I’m a converted believer in the possibilities of PDF translation… I’m not.  We’re unlikely to see the back of PDFs for translation any time soon, so I am happy to see the technology to support this workflow improving all the time.  I also don’t want to give the impression this is going to help with every PDF you ever see.  It won’t!  The problems of PDF quality don’t go away because of the way they been created in the first place, so source is always best.  You’re also quite likely to find PDFs you can’t handle even with Iris, and you might even find that the more basic option without Iris does a better job of your PDF conversion.  So it’s horses for courses… you have the tools and can apply the most appropriate one for your job.

If you have any questions after reading this post or watching the video then I’d recommend you visit the SDL Community and ask in there… or just post into the comments below.

A nice picture of a cutie cat… although I’m really looking for a cutie linguist and didn’t think it would be appropriate to share my vision for that!  More seriously the truth isn’t as risqué… I’m really after Qt Linguist.  Now maybe you come across this more often than I do so the solutions for dealing with files from the Qt product, often shared as *.TS files, may simply role off your tongue.  I think the first time I saw them I just looked at the format with a text editor, saw they looked pretty simple and created a custom filetype to deal with them in Studio 2009.  Since that date I’ve only been asked a handful of times so I don’t think about this a lot… in fact the cutie cat would get more attention!  But in the last few weeks I’ve been asked four times by different people and I’ve seen a question on proZ so I thought it may be worth looking a little deeper.

Read More

001Years ago, when I was still in the Army, there was a saying that we used to live by for routine inspections.  “If it looks right, it is right”… or perhaps more fittingly “bullshit baffles brains”.  These were really all about making sure that you knew what had to be addressed in order to satisfy an often trivial inspection, and to a large extent this approach worked as long as nobody dug a little deeper to get at the truth.  This approach is not limited to the Army however, and today it’s easy to create a polished website, make statements with plenty of smiling users, offer something for free and then share it all over social media.  But what is different today is that there is potential to reach tens of thousands of people and not all of them will dig a little deeper… so the potential for reward is high, and the potential for disappointment is similarly high.

Read More

001One of my favourite features in Studio 2017 is the filetype preview.  The time it can save when you are creating custom filetypes comes from the fun in using it.  I can fill out all the rules and switch between the preview and the rules editor without having to continually close the options, open the file, see if it worked and then close the file and go back to the options again… then repeat from the start… again… and again…   I guess it’s the little things that keep us happy!

I decided to look at this using a YAML file as this seems to be coming up quite a bit recently.  YAML, pronounced “Camel”, stands for “YAML Ain’t Markup Language” and I believe it’s a superset of the JSON format, but with the goal of making it more human readable.  The specification for YAML is here, YAML Specification, and to do a really thorough job I guess I could try and follow the rules set out.  But in practice I’ve found that creating a simple Regular Expression Delimited Text filetype based on the sample files I’ve seen has been the key to handling this format.  Looking ahead I think it would be useful to see a filetype created either as a plugin through the SDL AppStore, or within the core product just to make it easier for users not comfortable with creating their own filetypes.  But I digress…

Read More

001We all know, I think, that translating a PDF should be the last resort.  PDF stands for Portable Document Format and the reason they have this name is because they were intended for sharing with users on any platform irrespective of whether they owned the software used to create the original file or not.  Used to share so they could be read.  They were not intended to be editable, in fact the format is also used to make sure that the version you are reading can’t be edited.  So how did we go from this original idea to so many translators having to find ways to translate them?

I think there are probably a couple or three reasons for this.  First, the PDF might have been created using a piece of software that is not supported by the available translation tool technology and with no export/import capability.  Secondly, some clients can be very cautious (that’s the best word I can find for this!) about sharing the original file, especially when it contains confidential information.  So perhaps they mistakenly believe the translator will be able to handle the file without compromising the confidentiality, or perhaps they have been told that only the PDF can be shared and they lack the paygrade to make any other decision.  A third reason is the client may not be able to get their hands on the original file used to create the PDF.

Read More

001If you’ve never come across Microsoft Publisher before then here’s a neat explanation from wikipedia.

Microsoft Publisher is an entry-level desktop publishing application from Microsoft, differing from Microsoft Word in that the emphasis is placed on page layout and design rather than text composition and proofing.”

It’s actually quite a neat application for newbies to desktop publishing like me, but it’s a difficult tool to handle if you receive *.pub files (the format used by MS Publisher) and are asked to translate them.    And I do see requests from translators from time to time asking how they can handle them.  The file itself is a binary format and even with Office 2016 (which includes Publisher if you have the Professional version) the only export formats of PDF, XPS and HTML are not importable.  So very tricky indeed if you need to be able to provide your client with a translated version of the pub format.

Read More

01What the heck is a good bug? I don’t know if there is an official definition for this so I’m going to invent one.

An unintended positive side effect as a result of computer software not working as intended.

I reckon this is a fairly regular occurrence and I have definitely seen it before.  So for example, in an earlier version of Studio you could do a search and replace in the source and actually change the source content.  This was before “Edit source” was made available… sadly it was fixed pretty quickly and you can no longer do this unless you use the SDLXLIFF Toolkit or work in the SDLXLIFF directly with a text editor.  In the gaming world it happens all the time, possibly the most famous being the original Space Invaders game where the levels got faster and faster as you killed more aliens.  This was apparently not by design but it was the result of the processor speed being limited, so as you killed the aliens the number of graphics reduced and the rendering got faster and faster… now all games behave this way!  Another interesting example in the Linux/Unix world is using a dot at the start of a filename to hide it from view.  This was apparently a bug that was so useful it was never “fixed”.

Read More

%d bloggers like this: