Alright .NET gurus, solve this one Options

[Hartmann]

I have been scraping data off of a site (300KB) and they recently changed their URI string so that the date parameter has a hash of a date instead of the date itself.

The piece of the string I am looking at looks like this:

CODE

FDate=ZVbeOGQGDBnVmWDrJ6901w==&RDiret=Status

And more exactly the FDate parameter. Any ideas on the hashing mechanism? I know it can't be one way encryption because there would be no way to interpret that as a date (and they have to interpret it as a date).

[chook]

There is something similar i deal with dll files and their hashes. Have you seen if they do a lsb stegnography or something simple? I guess they would have to give you an interperter somewhere in code, or mabye you submit the hash to them as part of a script?

I know nothing of .NET, sorry.

[impala454]

I wouldn't rule out one-way MD5 or SHA1 or something. Their page could have a pre-defined set of dates which are hashed into a table that just sits in memory. Without knowing exactly what the page is or what it's doing it's difficult to tell. It doesn't really make sense to me though unless this thing is being called from some app or web service or something.

You can search the .net cryptography namespace on MSDN and see what turns up (System.Cryptography).

You might also try just picking a recent date (or one that makes sense in terms of the page), and use several different hashing algorithms and call the page/service with them and see what happens.

[Hartmann]

It wouldn't have to sit in a table as you are just using the parameter to confirm the date (i.e. if the hash matches the date criteria, it's allowed).

I figured out that they are using Base64. I decoded the Base64 and found binary data, I turned the binary data into hex and now I have a MD5 hash that I can't figure out.

[impala454]

I guess I'd have to know what the site is to undersand the meaning of those querystrings.

If the MD5 hash is just an encoded date and you have a reasonable idea of the format of the date, you can set up a quick program like this:

(obviously pseudocode)

CODE

string hashtodecode = ytghdgadoghdlurgh9438gh48hg
datettime date = startdate

while(date < enddate)
{
  date = date.adddays(1)
  if(getmd5hash(date.tostring) == hashtodecode)
        break
}

edit: you can do a quick google search for MD5 get hash C# or whatever language you're using to get the standard method everyone uses
edit2: you'll also have to play around with the date.tostring("yyyy/MM/dd"), with different formats until you guess it correctly

[Hartmann]

I guess I'd have to know what the site is to undersand the meaning of those querystrings.

If the MD5 hash is just an encoded date and you have a reasonable idea of the format of the date, you can set up a quick program like this:

(obviously pseudocode)

CODE

string hashtodecode = ytghdgadoghdlurgh9438gh48hg
datettime date = startdate

while(date < enddate)
{
  date = date.adddays(1)
  if(getmd5hash(date.tostring) == hashtodecode)
        break
}

edit: you can do a quick google search for MD5 get hash C# or whatever language you're using to get the standard method everyone uses

Well the issue is more the format of the date (MM/DD/YYYY, YYYMMDD, etc.) as each of those have a different representation when hashed by MD5.

Basically, the query string takes this parameter and compares it to a field in the database that holds a replica of it next to its actual value. It verifies that the data coming in is valid.

[impala454]

Well the issue is more the format of the date (MM/DD/YYYY, YYYMMDD, etc.) as each of those have a different representation when hashed by MD5.

right, you could just come up with about 10-15 different formats and run the loop for each. once you have the format you're set. that .net MD5 hash class is pretty damn fast so it shouldn't take long.