can I scrape web pages with AJAX on linode?

hi

is there any way I can scrape web pages that uses AJAX using something like ruby + mechanize on linode.

http://watir.com/ would be a solution but I guess not applicable to linode.

Thank you

R

4 Replies

Probably. Figure out what the application is doing to get the data you want, and then do that. AJAX isn't magic; there's still HTTP under the hood.

@hoopycat:

Probably. Figure out what the application is doing to get the data you want, and then do that. AJAX isn't magic; there's still HTTP under the hood.

sounds interesting!

I am not the author of the AJAX code so do you think that 'analysing HTTP' traffic let's say using firebug would do the job?

thank you

R

Perhaps. It depends a lot on the application and what you want to do… it can be a fun way to use tools in ways you don't normally use them, at the very least.

Firebug will probably get you moving; Wireshark is good to play with, as well.

@hoopycat:

Perhaps. It depends a lot on the application and what you want to do…
the first step will be to just to scrape (get) data, next one modify/enter data

@hoopycat:

change it can be a fun way to use tools in ways you don't normally use them, at the very least. yep, you're completely right. It sounds really exciting … also because I know nothing about it right now …

@hoopycat:

Firebug will probably get you moving; Wireshark is good to play with, as well.
I think I installed Wireshark once and didn't get how it works :-)

Thank you

R.

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct