[ 5 posts ]

Terry Wood

  • Offline
  • Profile

need suggestions for making js content searchable

Post Posted: Thu May 24, 2012 12:32 pm
+0-
For SEO purposes, I am in need of suggestions for enabling js-produced content to be searchable by googlebot.

see gist https://gist.github.com/2788703#comments
Environment:
SEO enabling service: EC2 instance hosts nodejs, request, jsdom, yui(3.6.0pr1)
target web site: remote linux server, yui3, php, mysql

Circumstances:
1. start with a functional web site built with YUI3 front-end and php/mysql back-end.
a. YUI3 typically requests content from back-end using YUI3 io module and renders the content for the user.
b. Typical page structure: HTML5, CSS, with javascript <script> tags with src= 'link' just before </body>.
Status
2. nodejs http.request responds with markup from a page of the web site.
3. jsdom produces dom of markup. jsdom defaultDocumentFeatures all false.
4. A new YUI instance is configured with the document and window produced from jsdom.
a. At this point, the YUI instance can manipulate the source code of the page.
b. The page source code looks fine when logged to console.
5. the dom manipulation logic is in the external js files listed in the source code before </body>.
6. The external js files can be loaded/read.
a. Y.Get.js returns onSuccess after js files are loaded.
b. nodejs http.request responds with js files.
c. other methods?
Suggestions for dom manipulation, please.
7. how can YUI apply the logic in those external js files to the source code?

Dav Glass

  • Username: davglass
  • Joined: Thu Aug 28, 2008 9:28 am
  • Posts: 2088
  • Location: Marion, IL, US
  • Twitter: davglass
  • GitHub: davglass
  • Gists: davglass
  • IRC: davglass
  • Offline
  • Profile
Tags:

Re: need suggestions for making js content searchable

Post Posted: Fri May 25, 2012 8:04 am
+0-
This is likely a JSDom issue and not a YUI issue. The documentElement is null after JSDom creates the DOM. This leads me to believe that it's failing to parse the DOM correctly and setup the needed documentElement reference.

Terry Wood

  • Offline
  • Profile
Tags:

Re: need suggestions for making js content searchable

Post Posted: Fri May 25, 2012 8:23 am
+0-
Thank you, Dav. I've tried to avoid use of jsdom beyond the initial dom creation. I had some hope that jsdom succeeded in creating a valid dom from the source code because Y.one('doc').get('outerHTML') produces seemingly good results. I'll look harder.

Your suggestions regarding item 7 would be much appreciated. I need help understanding how to apply the results of Y.Get(...external js) to manipulate a valid document and window in YUI.

Dav Glass

  • Username: davglass
  • Joined: Thu Aug 28, 2008 9:28 am
  • Posts: 2088
  • Location: Marion, IL, US
  • Twitter: davglass
  • GitHub: davglass
  • Gists: davglass
  • IRC: davglass
  • Offline
  • Profile
Tags:

Re: need suggestions for making js content searchable

Post Posted: Fri May 25, 2012 8:30 am
+0-
This is likely what would stop that from working:

Code:
FetchExternalResources   : false,
ProcessExternalResources : false,


Since the JS wouldn't execute inside the page.

On a side note, I'm not sure exactly what you are trying to accomplish here.

The main issue is that your site is not progressively enhanced, so if JS is not available (via older search bots or a browser with JS disabled or a mobile device over a really slow network) you see nothing? In that case, you should render the initial view of your pages on the server, then "enliven" them on the client with JS.

Terry Wood

  • Offline
  • Profile
Tags:

Re: need suggestions for making js content searchable

Post Posted: Fri May 25, 2012 9:17 am
+0-
Again, thank you.
I changed the jsdom defaults as you suggested. The gist shows the code and how it failed. Looks like a doctype is placed at the beginning of each external js. I did not expect that.

The original jsdom default settings were intentional because I want YUI to use the configured doc and win and external js to do all of the manipulation outside of jsdom. In other words, the only purpose of jsdom is to create a dom from the source without js.

Your assessment is correct that the site is not progressively enhanced. Given that situation, you are seeing my approach to add "searchability" to such a site. Googlebot will be instructed to see the searchable content rather than the source. https://developers.google.com/webmasters/ajax-crawling/
  [ 5 posts ]
Display posts from previous:  Sort by  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum