Skip to content
This repository has been archived by the owner on Mar 26, 2023. It is now read-only.

bing translator and non-well-formed html #75

Open
GoogleCodeExporter opened this issue Mar 14, 2015 · 24 comments
Open

bing translator and non-well-formed html #75

GoogleCodeExporter opened this issue Mar 14, 2015 · 24 comments

Comments

@GoogleCodeExporter
Copy link

Comment by mightysam, Nov 3:

We have been using the this jquery translation library with Google on our site 
for close to 2 years and it's working great. Recently we tried to switch to 
Bing and I can see the site start to fall part after switching between a few 
languages.

After some investigation, I believe the problem lies on Bing's requirement on 
sending well formatted HTML tags. 
http://msdn.microsoft.com/en-us/library/ff512406.aspx which may not be 
happening because jquery is breaking up the HTML.

For example, the jquery translation library may break up the site's HTML into 
the following before sending to Bing: </div><div>Text 1</div><div>Text 2</div>

Bing will return the following after translation, stripping out incomplete 
tags: <div>Text 1</div><div>Text 2</div>

When the library tries to put the results back to the original position, the 
website will be messed up with missing and misplaced tags. Any suggestion on 
how this can be fixed?

Original issue reported on code.google.com by balazs.endresz on 6 Nov 2011 at 4:02

@GoogleCodeExporter
Copy link
Author

Looks like it works if the text is translated as "text/plain": 
http://code.google.com/p/jquery-translate/source/diff?spec=svn242&old=241&r=242&
format=unidiff&path=%2Ftrunk%2Fsrc%2Fjquery.translate-core.js

Original comment by balazs.endresz on 6 Nov 2011 at 4:06

  • Changed state: Fixed

@GoogleCodeExporter
Copy link
Author

I have tried to switch to "text/plain" last week but it wasn't work that well. 
My page was broken down into 5 calls to the translation server, and the 2nd and 
3rd calls return empty result most of the time, thus messing up the page 
structure.

I will try to use your new file to see if that makes any difference.

Original comment by mighty...@gmail.com on 7 Nov 2011 at 8:51

@GoogleCodeExporter
Copy link
Author

Please let me know how it worked for you if you tried it. I don't have any time 
at all now, but will try to have a look at it before the weekend.

Original comment by balazs.endresz on 8 Nov 2011 at 6:42

  • Changed state: Started

@GoogleCodeExporter
Copy link
Author

I have tried the new library and I still have the same problem.
To investigate, I compare the calls that get sent to Google vs Microsoft and 
found that that the jquery-translate library is breaking up the page and calls 
differently.

For example:
Google: <div> </div>
Microsft: <div> </div>

Google: <div>Mar</div>
Microsoft: (1st call) <div>Ma   (2nd call) r</div>...

Is it expected that calls will be split up differently when using different 
service?

Original comment by mighty...@gmail.com on 9 Nov 2011 at 10:06

@GoogleCodeExporter
Copy link
Author

I don't really see how that could happen. Would you please send a test page by 
email, or somehow? I could help much more easily then.

Original comment by balazs.endresz on 10 Nov 2011 at 7:56

@GoogleCodeExporter
Copy link
Author

The Microsoft API seems to place HTML tags in the wrong place:

<div>Got a Q for 311? Now U Can Text It!</div>

is translated as:

Haben <div>eine Dame für 311? Jetzt u kann Text es!<\/div>

In some cases the exact position of tags in a sentence is not easily decidable 
but this looks like something that could be fixed easily by Microsoft.

In my experience the "text/plain" encoding works fine otherwise.

Original comment by balazs.endresz on 20 Nov 2011 at 7:11

@GoogleCodeExporter
Copy link
Author

But the tags are in the right place if I translate this with "text/html" but 
that can't handle non-well-formed HTML. So, you can choose which error you 
prefer :)

Original comment by balazs.endresz on 20 Nov 2011 at 7:23

@GoogleCodeExporter
Copy link
Author

If you use "text/html" with the MS API then apparently you can translate the 
whole page in a single request:
$("body").translate("en", "de", {limit:Infinity});

Google has a 2000 limit because of this: http://support.microsoft.com/kb/208427

Original comment by balazs.endresz on 20 Nov 2011 at 7:36

@GoogleCodeExporter
Copy link
Author

Yea, the tag will messed up if I use text/html.
But the problem I am facing is when using "text/plain", the library does not 
breakup the page correctly. Have you experienced that with the page I sent you?

I have tried to add the "limit:Infinity" but the page doesn't translate at all. 
To make sure it's not a format issue, I tried "limit:2000" and the page 
translates, although some parts are still messed up.

Original comment by mighty...@gmail.com on 2 Dec 2011 at 8:48

@GoogleCodeExporter
Copy link
Author

To sum this up, the problem is the following:

"text/plain": tags are misplaced

"text/html": HTML has to be well formed, additional tags are inserted

"limit:Infinity": it worked for me on the site you sent me but that isn't going 
to work on IE, so that's not good anyway, though I'll have another look at it 
when I'll have internet at home


Original comment by balazs.endresz on 3 Dec 2011 at 1:56

@GoogleCodeExporter
Copy link
Author

http://social.msdn.microsoft.com/Forums/en-US/microsofttranslator/thread/775fd59
2-9d40-4ca5-85a4-32ac39967c98

Original comment by balazs.endresz on 3 Dec 2011 at 2:02

@GoogleCodeExporter
Copy link
Author

Hello all, is jquery-translate latest version from trunk working with Microsoft 
Translator? There is any working example to use it? where do i need to put my 
appid and key?

thanks in advance, regards

Original comment by fid...@gmail.com on 6 Dec 2011 at 7:27

@GoogleCodeExporter
Copy link
Author

Hello developers, 

First of all thank you for best Project and excellent code.

With Bing I have much errors so I added some modification to code, now all look 
like all work.

developer may check self file with help Diff, and may be some thing write more 
best. Sorry not all place in code was clear.

1. In bing after translate I have white spaces between </div> and next <div> 
For solution I added to _process: function()
below row:
   this.rawTranslation = this.rawTranslation.replace(/<\/div>\W<div>/gim,'</div><div>') ;

2. some text was bad formated so I replace in function "_translate" line:
data: {appId: key, from: _this.from, to: _this.to, contentType: "text/plain", 
text: src },
to 
data: {appId: key, from: _this.from, to: _this.to, contentType: "text/html", 
text: src },

3. because all continue not work 
I return old version code in function "_process" I replace so line (may be I 
not right)
if(j++ < n)

replace to
  if(j < n)

4. All at top give some result but all continue not work.
so I go to function "_init"

And replace below code:
            substr = this.truncate( this.rawSource.substr(lastpos), o.limit);
             if(!substr) break;
             this.rawSources.push(substr);
             lastpos += substr.length;

to code:

            size_for_translat = 0 ;
            substr = "";
            for(var kk = 0; kk < this.source.length; kk++){
              text_for_translate = '<div>'+this.source[kk]+'</div>';
              len = text_for_translate.length ; 
              if( (size_for_translat + len ) < o.limit )  {
                size_for_translat = size_for_translat + len;
                substr += text_for_translate ; 
              } else {

                this.rawSources.push(substr);
                size_for_translat = len;
                substr = text_for_translate ; 
              }
            }

            if(substr) this.rawSources.push(substr);
            break;

after that all start work good. So may be 1. 2. 3. points may return to 
original state.


In attachment my file result:
Every developer may check self file with help Diff, and may be some thing write 
more best. 

At end small promotion: at ordasoft.com - you may download work version for 
site translate build on joomla CMS and yours code. Project sef Translate.

Thank you very much all !
Andrew


Original comment by akb...@gmail.com on 6 Dec 2011 at 8:08

Attachments:

@GoogleCodeExporter
Copy link
Author

@akbett: Thank you for the diff, I'll merge it on Saturday at last, I'm not 
going to have internet at home until next week.

Original comment by balazs.endresz on 7 Dec 2011 at 7:34

@GoogleCodeExporter
Copy link
Author

@akbett: 

Thanks Andrew!  I've seen much improvement with your fixes.  Thanks to everyone 
who's working on this lifesaver!

Original comment by elo...@gmail.com on 13 Dec 2011 at 4:44

@GoogleCodeExporter
Copy link
Author

I must say that Andrew has saved my life!  His changes made my totally screwed 
up looking Bing translated pages look normal again!

Just so people know the process, I grabbed the latest Source version from the 
Download Builder (Trunk). I was using 1.3.9 previously.

I'm also now loading jQuery 1.7.1 vs 1.3.2 previously.

I had to change the translate.ui syntax to be 1.4.x compliant.

I added our Bing AppID before running the jQuery.translate function and the 
translation worked for my app. However, as others have stated, the html was all 
messed-up.

I went through Andrew's changes and made them on my local version of the plugin 
source, ran it and it's all fixed!

Thank you Andrew!!

Original comment by rlindab...@gmail.com on 13 Dec 2011 at 9:31

@GoogleCodeExporter
Copy link
Author

Hi for this works ok , how i must put the code :


<script type="text/javascript" 
src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js"></script>
<script src="jquery.translate.js"></script>


And into the body tag -----

<script>
$("body").translate("en", "es", {limit:Infinity});
</script>

But no works and by other side where i must put the API -ID for use bing 

Can you put simple code for translate body of website ? , thank´s regards 







Original comment by sexcamtf...@gmail.com on 22 Dec 2011 at 5:03

@GoogleCodeExporter
Copy link
Author

Everything just stopped working a while ago, any news?

Original comment by eatf...@gmail.com on 19 Jan 2012 at 9:58

@GoogleCodeExporter
Copy link
Author

Hello there, at this point I must say: Thanks Andrew for Your effort and fixes 
in source code!

Do You have any problems with Internet Explorer 7 and this translate plugin 
(Bing version)? In wireshark I see that IE7 sends two request (last one to 
"V2/Ajax.svc/GetLanguageNames"), get HTTP 200OK and that's it - nothing else 
happens...

Do Andrew, or anyone, knows where I should search for problem in source code?

Original comment by krypton...@gmail.com on 23 Jan 2012 at 11:01

@GoogleCodeExporter
Copy link
Author

[deleted comment]

@GoogleCodeExporter
Copy link
Author

Ok, I solved problem with IE7. I saw that ajax request, which handles main 
translating work (in Bing option it is that with URL 
http://api.microsofttranslator.com/V2/Ajax.svc/Translate), is not send because 
of... too long GET request (and I'm translating a lot of text).

Solution: simply add limit to Your translate options, for example "limit: 1000" 
(default if 1750).

Original comment by krypton...@gmail.com on 26 Jan 2012 at 10:55

@GoogleCodeExporter
Copy link
Author

Hi,
I use jquery translete only for plain text and i run in this example:
$.translate( 'Šiaudų granulės ', { 
          from:'lt', 
          to:'en', 
          complete: function(translation){
                console.log(translation);
            }
    });

It seems that bing for this text in example (i didn't found more examples) 
wrapped i <div></div> returns "</div> <div> Straw pellets" which freezes 
browser. I use this plugin only for translating plain text so i just changed 
"text/plain" to "text/html" and i works fine now. 
For anybody who has similar problem (his browser freezees or crushes) and 
changing to "text/html" doesn't work for him i think that the problem lies in 
_process: function but i did't look closer to this.  

Original comment by lsobo...@gmail.com on 2 Mar 2012 at 10:21

@GoogleCodeExporter
Copy link
Author

I looked closer to this and i think that there is infinite loop for string 
"</div> <div> Straw pellets" in this fragment of _process function:

while( (lastpos = this.rawTranslation.lastIndexOf("</div>", i)) > -1){
            i = lastpos - 1; // i==-1
            subst = this.rawTranslation.substr(0, i + 1); // subst == ""

            divst = subst.match(/<div[> ]/gi);  // divst == null
            divcl = subst.match(/<\/div>/gi); // divcl == null

            divst = divst ? divst.length : 0; // divst == 0
            divcl = divcl ? divcl.length : 0; // divcl == 0

            if(divst !== divcl + 1) continue; // 0 !== 1 ; infinite loop starts
...

Original comment by lsobo...@gmail.com on 2 Mar 2012 at 10:55

@GoogleCodeExporter
Copy link
Author

Hi to all,

is there a guide what i must change to work the translation? 

I am using it to this website http://www.psarema-skafos.gr 

My code was

function translateTo( destLang ){
  $('body').translate('el',destLang, {
      not:          '.jq-translate-ui',
      start:        function(){$('#translate_loading').show();},
      complete:     function(){$('#translate_loading').hide();},
          fromOriginal:true
      }
  );
  $('title').translate(destLang)
}

$(document).ready(function(){
        $.translate(function(){
        $('#translate_loading').hide();
        $.translate.ui({
        tags: ['select', 'option'],
        filter: ["el","en", "de", "fr","it","tr"],
                label: $.translate.toNativeLanguage ||
         function(langCode, lang){
           return $.translate.capitalize(lang);
         },
                fromOriginal:true
     })
     .change(function(){
                var lang = $(this).val();
                translateTo(lang);
        $.cookie('destLang', lang, {path:'/'});
        })
        .val('el')
        .appendTo('#translate'); //insert the dropdown to the page
    $.translate.getBranding().appendTo($('#translate'));
        var destLang = $.cookie('destLang');
        if (destLang && destLang != 'el') {
                //alert(destLang);
                $('.jq-translate-ui').val(destLang);
                translateTo( destLang );
     }
    });

});

So i must include the modified version of the plugin for the Bing API. 

And what else; 

Thanks, Michael



Original comment by mbas...@gmail.com on 4 Apr 2012 at 11:01

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant