Old news

I've received 3 corrupt X files so far:

• Two seem to have single-bytes with values which are simply incorrect according to my findings. Theoretically both of these could be fixed by an automatic tool. They could also have been avoided if the player used 2.6jrc4, available to download from Stars! Authost's download section. The only difference between the 2.6 (registered shareware) and 2.7 (retail) versions that I am aware of are that the 2.6s don't have battle board sound effects (boohoo!), and they have a different splash screen. The same serial numbers work for both.
• The third corrupt X file is from an AI, and it's a very interesting one; somehow an order from another player's X file managed to find it's way into the AI's X file. Not entirely surprisingly, this order is not at all valid in the context of the AI player's turn, and thus, once the X file has been generated, stars! balks at generating the turn - regardless of OS or version. Why do I even mention the OS? Because the corrupt X file is generated consistently... as long as the generation attempt is done on XP. Try it on 98 (without a pre-existing corrupt X file), and a valid X file is generated. Most mysterious.

I'm working on a tool similar to the race checker which will check and possibly even repair problematic X files; however, these are relatively rare and so far I only have two samples to work with. There seem to be at least two separate categories of problems with X files - one related to compatibility between JRC3 and JRC4, and one occuring even within JRC4. I may have worked out the first of these problems, and am working on the second - but I need more data for both! If you have corrupt X files, please send them to me, with as much accompanying information as possible - in paticular, the accompanying XY and M files are invaluable, and if the HST file is available, that would be very helpful too.

Discussion on Autohost (Hosting forum, autohost login required to read / post).

I've put up a simple web service to check and optionally repair race files; it's available at http://wumpus.mine.nu/stars/check_race.php.

It's running on my personal web server at the moment, and thus will occasionally be down. Trying again 5 minutes later is unlikely to help. If it's down for more than a few hours, you can e-mail me.

Feedback is welcome, as usual.

Discussion on Autohost (Hosting forum, autohost login required to read / post).

I've put initial mockups for the H file combiner web front end up here. Use the links at the top to navigate. The session management has kind of worked, but didn't automagically on SF - probably because it wants to write in /tmp and SF doesn't like that, or similar. In any case, I disabled the need to be "logged in" (with any username or password :P) - just click on the other links at the top and they will work (insofar as mockups "work"). It's not supposed to run on SF in the long run anyway..

It should give you an idea of the way I imagine presenting the web front end for this tool - if you think it sucks (for the presentation or for the basic concept), then feel free to come up with something better. I'm not a huge fan of website design :P (What, this page already gave you a hint ? Damn :P)

If someone feels inspired by the front end and wants to work on the backend, I'd be more than happy to accept help there too.

In any of the above cases, just e-mail me - as usual :-)

I just scanned over the page and noticed that, even reading the AH thread linked some time ago, it's not actually clear what it is that I'm working on, so, here's a quick explanation:

• The original objective of the project was to pool source code and source code snippets for various Stars! utilities, to save people wanting to do similar work from reinventing the wheel.
• This purpose is theoretically still alive and well, but I haven't pursued it actively.
• I decided to make the first step myself by documenting the XY file format insofar as it was not encrypted. The documentation hasn't yet been linked yet, sorry - although of course the source of leit's tool, based on my notes, could be considered "documentation".
• I got a bit carried away investigating the encrypted parts of the XY files. Somewhere along the way the encryption method (given key and plaintext, generate the cyphertext) became apparent.
• A lot of number crunching later, I was able to decrypt almost any stars! player (X/M/R/H/XY) file (no details of how yet, sorry :P).
• Since then I've been working on deciphering the data unencrypted data structures, and putting this knowledge to at least some good use, although admitedly not much to (publically) show for it.
• In future? Who knows.
  • As mentioned earlier, I have a utility (with a horrible interface) to merge knowledge from multiple H files to let players share knowledge of the universe which they can then see "in game" rather than in an external utility.
  • Also mentioned earlier, I'm working on a tool to check submitted X files for various more or less subtle cheats which the main stars! hosting utility does not pick up.
  • User-written AIs would certainly be conceivable, although I would consider this Really Hard. Player aids (eg: optimize exploration of the rest of the universe using these 5 scouting ships) could be a nice starting point though.
  • A complete server and/or client drop-in replacement for the original stars! binaries would certainly be conceivable... If so, I'd like to work together with the freestars project rather than each of us doing their own thing.
  • Special "scenario" games could be done; I've already helped set up one or two special games hosted on AH; One could even consider scripted games (in paticular turns, some "random" events occur - from mineral fountains to comets to Mystery Traders to population explosions...); it's even conceivable to have a dynamic map (EG: every 10 turns, the known galaxy expands by, say, 2%, or planets disappear or appear, or...).
  • Who knows what lese one might be able to do :-)

The ais.txt file has been updated with details of the easy and regular AIs; since I only did these recently, the information missing in the cases of the tough and exepert AIs is actually present here.

Some time ago, I patched together an H file combiner and beta tested it for some teams in the ringsm game. It became moderately polished in that time, and now only really lacks a web service front end (I still don't feel able to release the binaries, let alone the source - see the update on the 10.10.2004). But that's work I find interminably boring, so the X file checker mentioned in the update below is going to get more attention for a while.

If someone feels like working on this, I can write up a spec for what I need; it's fairly simple... Just e-mail me at Michael "Wumpus" Zinn <das.wumpus@gmail.com>.

Something that came up out of a recent discussion on autohost is the question of the AI race designs... So here's details on the expert and tough AIs, as I determined them some time ago.

So, in the last few months, has there been much progress? Well.... no. But since autohost has recently added some password protection to M file downloads, one of my main concerns with the viability of putting this work in "the public eye", as open source, is at least partially addressed, so work has sporadically resumed.

My immediate objective is to write a basic validity checker of X files - given an HST file and an X file, the checker should catch a few tricks which slip by the stars! hosting utility. EG: Prior to 2.7jrc4, the validator could have checked for the freepop hack. I don't immediately intend to check for "shades of grey" things (split fleet dodge, N/S or E/W minefleet immunity), but I will aim to catch some of the more clear cut stuff (quick starbases, and some nasty X file tricks which - as far as I know - aren't common knowledge... yet). Most of this is fairly trivial, but there are some considerations about how whole universes should be stored in memory I should Do Right the first time, rather than making a quick hack now and getting a headache over it later.

As usual, don't hold your breath... "real" (ie: paid :-P) work (fulltime plus a sideline) is keeping me more than adequately busy...

First link: the planet names list mentioned in the last update: planetnames.c (fontified) or planetnames.c (plain). Sorry this wasn't linked sooner.

Second link: Map2XY download. Another one that has been up for a while.

And lastly: In spite of appearances, a lot has happened in the last 2-and-a-half months. But none of this has been uploaded, even into CVS, for reasons that you might be able to conclude from reading this thread on the Stars! Autohost forums. (Yes, I know, subtlety has never been my strong point, at least not in that kind of thing).

OK, I've uploaded my tamperings with the xy file into a new CVS module, xyfile.

• planetnames.c/h - a complete list of the planet names in the un-hacked stars! (as of 2.7jrc4), in their "name id number" order. Yes, the list is nearly, but not perfectly, alphabetically sorted... probably some last minute changes by the Jeffs came in there :)

• xytomap - read an xy file and spit out a map file, as per "Dump Universe Definition" in the in-game menus, or the appropriate command-line parameter to the game. Not terriby useful in its own right, but it is complete (as far as I know). NB when run on a unix system, this generates files identical to what stars! produces. If run on a windows system, the EOL string probably needs to be redefined. NB2: The xy file is read from standard input!

• xyread - as above, but dump info on the headers and "mystery" bits of the file. Mostly for analysis purposes, at the moment.

Welcome to LEit, a new developer who is on board. He'll be adding 'map2xy' in the next few days, and I'll throw together the few additional "known" or reasonably hypothesised ideas about the XY file format that I've gathered.

Is there hope after all that this project isn't completely dead?

Ahem. Right. I'm getting back up to speed again. Hopefully there'll be some updates over the coming weeks. The project hasn't been forgotten about. Honest!

Nothing too exciting here yet; but there's a sample of some of what the script(s) do so far here. This is from a team game which is now over.

Since there's not a great deal in the way of 'instructions' on the report page, here's some points on what's so exciting:

• The combined fleet report
  • All fleets are reported exactly once, provided any of the contributing players' can see it. IE, the combined report will never show the same fleet twice.
  • The owner's report of a fleet is chosen in preference to another contributor's report of a fleet. IE, the Cramontini reports on Cramontini fleets are used, in preference to Ork or Badger reports on Cramontini fleets.
  • When two contributing reports both show the same enemy fleet, but one has "better" data (IE, knows the actual most-signficant design name), it would be nice to use this in preference to the "less complete" data. Note that this becomes difficult - probably impossible given the data from the text reports generated by Stars! alone - when an enemy design is replaced, and the two contributing reports have conflicting non-default design names. Something that might be worth considering in this case, is to use a Warmonger's reports in preference to anyone elses... So the priorities might be: owner > warmonger > non-default fleet name > default fleet name. This much should be doable, and indeed not mindblowingly difficult, with the information currently available in the DB.
• Planet reports: A fairly obvious one again, but there are some things worth noting:
  • The "best" planet reports from all contributors are used; the heuristics for determining "best" here are moderately good now:
    1. If available, the planet owner's report is used
    2. Failing that, the most recent available penetrating scan is used.
    3. Failing that, the most recent non-penetrating scan is used.
    Note that non-penetrating planet scans don't really crop up all that often - they only crop up when a non-scanner equipped fleet enters orbit of a planet which hasn't been visited at all yet by that player, OR a scanner-equipped ship is shotdown before it can send a "full" report. Note that all scanners are 'penetrating' when the carrying ship is in orbit.
  • A "best guess" of ech planet's habitability for the given race is calculated. This is calculated using rather messy formulas that some enterprising souls on the rec.games.computer.stars newsgroup worked out some time ago. Its not perfect, but seems to be at worst within 2% of the right value.
  • Similarly, a "best guess" of the terraformed habitability of each planet for the given race is calculated, using the race's current terraforming technology. Total Terraforming is correctly taken into account. Unfortunately, this relies on the tech level table in the database being manually updated regularly, since AFAIK there's no way of determining tech levels from the reports - except possibly by doing some pretty heavy analysis of the terraformed habitability values in the reports produced by stars itself - but that, I suspect, would be more than just ugly.
• Hab images: The current centrepiece :)
  • The colour of a planet's name indicates the most-recent known owner (white if none or unexplored).
  • The size and colour of the planet itself indicates its habitability.
    • Planets with unknown surface conditions are white.
    • Planets which cannot, with current terraforming technology, be made habitable, are a fixed radius red circle. If someone really wants it, it shouldn't be too hard to make the radius of the red dot dependent on just how hostile the planetary conditions are (just involves plugging the appropriate (simple) formula into the hab analyzer generating the planet reports - its not the same as for habitable worlds. The changes to the image-generating script are trivial).
    • Planets which are uninhabitable unterraformed, but can be made habitable, are a blue dot, with the radius directly proportional to the terraformed habitability. Note that its blue rather than yellow, because I'm partially colour blind and find the yellow and green used by stars very hard to distinguish... of course I could just use a darker green :P This really should be configurable anyway (its currently hard-coded in a PERL script - trivial to change for a coder, but not for an end user.. :P)
    • Planets which are habitable without terraforming, are green with a radius directly proportional to their current habitability. If terraforming can improve this, then there is surrounding blue ring, with the outer radius indicating how large the green circle would be after maximum (current) terraforming tech was applied.

Unfortunately a number of this still require manual entry into the DB - such as race information.

Also, more configurable things should be moved into the DB - some things still are (EG, the xfig colours to be used for each race - this is why, in the example, the Cramontini/Ork/Badger planet names are all shades of blue, etc), but more should be.

One day, there should be a gui on all this :) But thats far in the future at this point, I suspect...

Interested in having a closer look and/or hacking around a bit on this yourself? Feel free to grab the code, of course (I'll dig around for anonymous CVS instructions for SF when I have some time - and I suppose I should think about setting up packages at some stage), and if you want to contribute something, get in touch with me: Michael "Wumpus" Zinn <michaelz_wumpus@users.sf.net> (German or English. You can try other languages too, but I probably won't understand a word and discard your mail on the assumption that its spam ,-P)