I've reverse engineered AcrossLite.puz *cackle*

[kelvin]

I think I may yet enter uDG this year (no promises though). Today I finished re-implementing the file checksum for AcrossLite (http://www.litsoft.com/AcrossJava/index.html)'s .puz file format.

If I do enter uDG with my xword game, then it'll support standard AcrossLite .puz files. :cool: In any event, I have a file integrity check for .puz which will make possible the creation of a new xword puzzle generator for Mac OS X!

Currently even the #1 xword puzzle app for Mac OS X (http://www.advenio.com/macxword/index.html) does not support the AcrossLite checksum! The only app that supports the checksum is LitSoft's own AcrossLite (and it only runs in Classic, yuk).

Okay, rant, rant. But what good is all this you ask? I have no intention of publishing my own xword app to compete with the likes of Advenio. What I'd like to get out of this is some recognition so I can perhaps land myself an internship. As far as re-engineering software goes, I've got Safari's WebFileDB and now this under my belt. Surely someone out there (who can employ me) appreciates this somehow...

How did I do it?
Well... having no formal training in these matters, I did what I did for SWFdb. I shoved a whole bunch of data into AcrossLite (I hate classic btw) by hand and then pounded the resultant output to a pulp with my brain. Basically, I stepped through the code backwards, in my imagination, blindfolded, and barefoot. Lots of adding, subtracting, and bitshifting later, I have 5 lines of code that faithfully reproduces the correct AcrossLite checksum with a given xword puzzle data file. It's even forward compatible as I respected AcrossLite's serialized field delimiters. :D

So, where am I going from here?
Well, most immediately, I can use the code I just wrote to make that xword game I was muttering about for uDG (school and work permitting). In the long run, I think this was good exercise for my brain. I haven't learned assembly yet but I'm thinking I should soon so I can keep going in the direction I'm heading. I'll have to get to know gcc a little better...

Yeah. That's about it. Not that any of this was relevant or important, but I just spent the greater portion of my free time over the last few days solving this puzzle. And I like solving puzzles, so satisfaction is high. :rolleyes:

I hope every body has a good week/month/year/decade,

[codemattic]

sweet hacking job Kelvin!

[skyhawk]

I'll have to get to know gcc a little better...

I think you mean gdb

[kelvin]

No, learning assembly is gcc. :p

[codemattic]

ok - now the question is do you feel its ethical to release it? The company (as I understand it) makes its money by paying people to design cw puzzles and then licenses those out in an applet or app form. If you provide people with a shortcut you take bread off of their table. Doesnt this seem similar to hackers who reverse engineer a shareware's registration scheme and posts a workaround?

[kelvin]

I read the license that came with the AcrossLite client I downloaded from their site.

In it there are provisions that forbid the following:
1) Decompiling/Disassembling/ReverseEngineer the client software object code.
2) Including any part of the client software in a derivative work.
3) Use the software for a commercial gain.

None of which I have done. What I did was to write code that reads and writes the same files.

My code uses no part of the AcrossLite object code.
I did not decompile or read any of the object code.
The code that I wrote does not use or rely on any of the AcrossLite software to do it's thing. :ninja:

What I've done is analyze files used by AcrossLite and write software that has a similar behavior. Kinda like how OpenOffice or TextEdit can read/write MSWord documents; neither relies on, or is derived from, MSWord. :D

comments?

[codemattic]

IANAL - so I dont know the legality of it.

The value in MSWord is the functionality the app provides - the file format provides a kind of lock-in which makes their customers less able to easily move to another competing app. It seems fair if someone takes the effort to program an app with word processor functionality that they can import/export other file formats to remove the lock-in. The value there is in the programs themselves.

The value in the puzzle example is stored in the puzzle files themselves - not in the program really. The big continuous effort and investment is in continually thinking up and designing new puzzles.

In the MSWord example if somebody made a better app and sold it cheaper they would deserve to win (even though most people for some reason cant imagine moving from MS Office) - the business would move to them - people would still have a word processor to use.

In the puzzle example - if the puzzle designers are unable to encrypt the puzzles and make sure they get paid then briefly everyone has access to fun puzzles - until the puzzle authors call it quits and then nobody has access to them.

I see a moral difference between the two.

However - it still is a very impressive hacking job!

[kelvin]

Currently, the acrosslite format is use by most major news publishers (NYT, washpost, etc.). I highly doubt they will stop making puzzles because litsoft's client is no longer unique. LitSoft hasn't updated AcrossLite since Mac OS 8.2. I'd think the format is in danger more if I didn't write my software.

My software doesn't doesn't prevent any of the publishers from getting paid, all it does is make publishing available to everyone. Currently, if you wanted to publish your own xwords, you'd have to use LitSoft's client to encode. The license specifies that you can't sell the xwords you create with the client. My software lifts that restriction by offering a competing option. The only other way to sell your own xwords would be to get a publishing version/license of LitSoft's client which, as far as I can tell, is impossible since nobody at litsoft will respond to emails or even administer their website.

All that aside, thanks again for the comments!

[Josh]

LitSoft hasn't updated AcrossLite since Mac OS 8.2.Who knew 8.2 even existed? :p

[kelvin]

Who knew 8.2 even existed? :p That should be 8.5 :P. Stupid number pad. :mad:

[geewhiz]

Litsoft has just released V2 of Across Lite, available here:

http://www.nytimes.com/ref/membercenter/help/litsoft.html

This is a free version for Mac OS 9.x and OS X, version 10.2 and higher (344K)
Mac OS 10.0 and 10.1 can run Across Lite in "Classic Mode" (Of course, other operating systems are supported as well).

This new version allows entry of multiple characters in cells (rebus) and also more than just alphabetic entries.

Litsoft does not charge constructors to make puzzles. They make their money by licensing the software to companies so that puzzles can be uploaded to websites in an interactive form.

[kelvin]

cool, should make my dev effort a bit faster as I won't have to bootup classic. Upon preliminary inspection of the files I've generated, they are version "1.3" (up from 1.2) and aside from a few new features, have not changed drastically. :)

[litsoft]

Just stumbled on this.

First, we are impressed by the reverse engineering. We can only imagine the bits and byte twiddling you must have gone through.

Just a few points from our side:

The checksums were not designed to make the format a secret but rather the file to have easily checked integrity and to prevent corrupted files from making our clients misbehave which would have prevented widespread distribution and usage of the program. If there is one reason Across Lite is popular with people, it is that it works robustly on every crossword created using our software (which have been all of them at least so far) and allows people to convert to our format very easily for both commercial (with licensing) and non-commercial use.

Our business model is based on free clients and licensing to commercial providers. In the niche crossword market, the other way wouldn't work because enough people wouldn't buy a program to encourage content providers to even bother with this form of distribution. We have made it easy to convert puzzles to our format using our client although software to help one create crosswords (crossword compiler) is not our main strength and we expect others to do so.

We keep the binary format (as opposed to the text format) unpublished/open is for two reasons. First, if people started creating these files on their own and screwed up, proliferation of bad or corrupt puzzles would have come in the way of a widespread distribution and usage of our clients and more annoyingly would have resulted in a lot of support requests that cost us for following up until we figured out the problem was with the third-party generated crossword. So it was a way to control one aspect of it, to enable the business model of a freely distributed client. The other problem, of course, is people creating malicious files with the right checksums to exploit any vulnerabilities in the program (try to induce stack overflows, etc.). Of course, one could say the program must be made bullet-proof but as people in this forum know very well, there is no such thing, it is just a question of who has more persistence. Relative safety from malicious content is another reason the program has been popular especially in a downloadable program that can potentially cause a lot of damage running locally on a computer.

Our competition is not people who write programs to create crosswords but rather people who write other clients. It is very easy for people to write clients for our formats as the main content of the crossword is kept in clear, unencrypted text and there are a large number of such clients out there, especially for mobile platforms.

For the people who want to write crosword compilers (i.e., programs that help people create crosswords) and so need to write files in our format, our approach is to come to an agreement to provide libraries that could be incorporated into their programs to export to our format. That is a way to ensure that the created crosswords are not malicious. But we have had to be selective about who we made agreements with because it involves some resources on our part to help them incorporate the library. We have thought about placing these libraries in the public domain with a self-supported community for support but this is a niche market and there isn't enough of a community to self-sustain it. So we only worked with people that had written widely used programs and were focusing on complementary crossword compilers rather than competing clients and so we cannot do this with every project out there.

In terms of legality, as long as our crossword program was used in accordance with the license, we have no problem with it. If any such usage helped reverse engineer the format, there may be a gray area if the result (the reverse-engineered format) of using the program is leveraged for commercial use but that is for the lawyers to worry about and that is worthwhile only if it results in a significant commercial loss. The format is also copyrighted which has some protection (to be used in case of abuse of our client or business model).

We do not worry about people writing competing clients to ours or creating home projects for non-commercial use. But if there is an attempt to leverage the availability of our clients (costing us directly/indirectly in terms of support/maintanence) to support a commercial business model for themselves, then there would be both an ethical and legal problem to be solved.

As a bottom line, we don't necesarily see any ethical or legal problems in the reverse-engineering itself. We believe that as long as the people recognize our intent and approach and lack of any predatory intent in our decisions in this niche market, they will not attempt any abusive use of that knowledge. Hence this post.

Across Lite itself is built with an architecture that allows a lot of plugins to be written by third parties to customize, and incorporate a lot of new functionality that should be more fun for the developer community than the basic display of grid and clues. We just have not had the time/resources to openly publish this architecture and documentation yet.

Walter Mason

[kelvin]

Alrighty, if I ever release anything, I'll give you the heads up. In the mean time it's all just for fun. Like I said, it's all just for my enjoyment/torture. :cool: