If I have
example code:

url = "someurl"
values = {"username" : USERNAME, 
          "password" : PASSWORD}
data = urllib.parse.urlencode(values)
req = urllib.request.Request(url, data)
urllib.request.urlopen(req)

it passes username and password to the url that you request.

So I want to post a thread in a forum. When I look at the pages source I see

<input id='topic_title' class='input_text' type="text" size="50" maxlength="150" name="TopicTitle" value="" tabindex="0" />

Im not too familiar with http so what is the key I want to use for the values dict for that line?
topic_title? or input_text?... I dont wanna brute force it because people might get angry at like 20 test threads lol.

I help with a game after everytime I play said game I have to post a replay, it's sort of tedious so I wanna automate it.

edit:
<input type='button' id='add_files_attach_0' class='input_submit' value='Attach This File' style='display: none; clear: both' tabindex='1' /> <span class='desc' id='space_info_attach_0'>Used <strong>9.45MB</strong> of your <strong>976.56MB</strong> global upload quota (Max. single file size: <strong>256MB</strong>)

This is the source line for attaching files how would I attach them?

Recommended Answers

All 3 Replies

Im not too familiar with http so what is the key I want to use for the values dict for that line?
topic_title? or input_text?... I dont wanna brute force it because people might get angry at like 20 test threads lol.

the Dict keys are

<input id='topic_title' class='input_text' type="text" size="50" maxlength="150" name="TopicTitle" value="" tabindex="0" /> 

url{"id":"topic_file","class":"input_text"...}

You got the idea??
:)

Oh forgot about this thread. Anyways yes, I was eventually able to figure it out.

Here was my follow up question on another site, I haven't been able to get an answer from there.

So there's two things that we do on a certain site, which is download new files and post files, and they're easily definable (and tedious), so I've been trying to script it.

So anyways, I have the file downloading script done. I'm able to re use the login code

#Cookies
    cookies = http.cookiejar.LWPCookieJar()
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cookies))
    urllib.request.install_opener(opener)
    
    #Authenticate user
    print("logging in")
    url = "http://someurl.com/index.php?app=core&module=global&section=login&do=process"
    values = {"username" : USERNAME, 
              "password" : PASSWORD}
    data = urllib.parse.urlencode(values)
    req = urllib.request.Request(url, data)
    urllib.request.urlopen(req)

But to post a thread with said file you have to attach it. I know how to send it the proper title and text for the thread from what I learned on how to log in. My problem is that to attach files you have to send a request to a **form** not a page request.
E.G. You select what file you want from file dialog and then click attach files, which it then uploads, you then finish writing up you're thread and THEN submit page.

Here's relevant html

<fieldset class='attachments'> 
     
    <script type='text/javascript'> 
    //<![CDATA[
     ipb.lang['used_space'] = "Used <strong>[used]</strong> of your <strong>[total]</strong> global upload quota (Max. single file size: <strong>256MB</strong>)";
    //]]>
    </script> 
     
    <h3 class='bar'>Attachments</h3> 
    <!--SKINNOTE: traditional uploader needs this. --> 
    <div id='attach_error_box' class='message error' style='display:none'></div> 
    <input type='file' id='nojs_attach_0_1' class='input_upload' name='FILE_UPLOAD' tabindex='1' /> 
    <input type='file' id='nojs_attach_0_2' class='input_upload' name='FILE_UPLOAD' tabindex='1' /> 
     
    <ul id='attachments'><li style='display: none'></li></ul> 
    <br /> 
    <span id='buttonPlaceholder'></span> 
    <input type='button' id='add_files_attach_0' class='input_submit' value='Attach This File' style='display: none; clear: both' tabindex='1' /> <span class='desc' id='space_info_attach_0'>Used <strong>9.45MB</strong> of your <strong>976.56MB</strong> global upload quota (Max. single file size: <strong>256MB</strong>)</span>

I have no idea how to code this, so I'm looking for direction.

Also on a sidenote if this sorta script might've been easier in other languages tell me which? I only used python because I knew it.

Thanks a lot.

One person gave this as feedback

"Just FYI: Forms submissions are page requests, and when you do your login, since you are using a data value in your Request() call you are sending a post request to a form already. The problem is figuring out how to send the file, to what URL etc, which can be tricky since it's done via javascript"

How would I find out how where send the file?

Thanks again.

P.S. Also no offense but you're wrong. The dict keys would simply be

values = {"input_title" : "this is the title I want"}

Log in that use Javascript make the process a lot harder.
Read Embedded script is messing up my web-scraping
You have Spidermonkey

"Execute arbitrary JavaScript code from Python.
Allows you to reference arbitrary Python objects and functions in the JavaScript VM"

Look at PyPi for tool that can help.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.