Neal Grosskopf Dot Com A NG Designs Site

View Link Entry

• Actions
  • New Open Link
  • View All
  • Search
  • Bookmarklet
  • Subscribe to RSS Feed

Web

( Add Link )

Googlebot Now Crawls Through HTML Forms

4/17/2008

From the Google webmaster blog, apparently Google now crawls forms and javascript on "some websites." I was a bit concerned about this at first because I don't want Google posting on all my forms with useless spammer like posts. Apparently they are only crawling GET methods and not POST. Either way, make sure you aren't using GET to add anything to a database or textfile.

By: Neal Grosskopf

Tags: google ( Add Tag )

Replies:

• 

  Chris Retlich
  #1

  Fri. April 18 2008, 2:47:25 PM

  Well, GET is supposed to be stateless, whereas POST isn't (of course developers make this mistake fairly often, linky). So it's not supposed to break anything by sending random GET requests, assuming that the developers wrote their stuff correctly. GET methods are useful for searches, and for showing different blog posts, etc., and those kinds of applications wouldn't be broken by random bot-driven GET requests, while POST is stateful or whatever, and are for "changing" something on the server, e.g. login, submitting comments, etc. Of course, GET only changes the URL, while POST doesn't, so that distinction is important too... never use hyperlinks for something that changes something on the server! Enough rambling from me for now...

• 

  Neal Grosskopf
  #2

  Fri. April 18 2008, 9:29:16 PM

  I usually do a lot of POST forms with GET methods i.e method="submit=true". I think I initially learned that trick from the apps on Lakelands site i.e id=1 id=2 etc. It sucks not having an uber programming genius like you around, I get sort of lazy with my techniques because I don't have you hounding on me to get better!

• 

  Chris Retlich
  #3

  Sun. April 20 2008, 5:06:52 PM

  I've done a bunch of mixed forms like that too. Not sure why exactly. Some places, and maybe a program or two) I have seen dislike that method, not sure what the "official" stance is on it. It is a nice shortcut instead of having to add <input type="hidden" /> tags to your forms.

• 

  Chris Retlich
  #4

  Sun. April 20 2008, 10:23:26 PM

  The Spider of Doom and The Spider of Doom comments, from my new favorite website, of course. This article was linked to from the /. discussion of the new Googlebot behavior.

• 

  Neal Grosskopf
  #5

  Mon. April 21 2008, 7:07:45 AM

  I've always liked using it because I don't need to use a separate page after each POST. Unfortunately at Alliance almost all our pages have a separate .asp page for each step of the post, ie. form page, process page, user response page, even though that could easily be put on one page.