w00zla Posted March 5, 2017 Share Posted March 5, 2017 Good news everyone ;D I found a way to read the sector XML files in an usable way within ingame scripts :) This means i managed to create a mod which can be used to search for specific stations in every found/created sector in your galaxy! SCREENSHOTS: Search UI Search Command The mod stays alpha until sector data is provided by game API! This implementation is a workaround until proper access to sector data is provided by the game for scripts, and some functions used for file access by this script even may be permitted in the future due to security/performance reasons (mods being able to read/write any file on the system is never good in terms of security)! BACKUP YOUR FILES/GALAXY! FINDSTATION v0.6alpha: Commands: /findstationui Enables/disables the UI (menu item & window) for station search. Usage: /findstationui /findstationui enable /findstationui disable /findstation Finds near stations in any of the found/created sectors in the galaxy and displays them in chat-window. Usage: /findstation <SEARCHTERM> Parameters: <SEARCHTERM> = term to search in station names (spaces possible, case-insensitive) /findstationconfig Used to set the configuration values for /findstation command. Usage: /findstationconfig galaxy <GALAXYNAME> /findstationconfig galaxypath <GALAXYPATH> /findstationconfig searchmode <MODE> /findstationconfig maxresults <NUMBER> /findstationconfig framesectorloads <NUMBER> /findstationconfig maxconcurrent <NUMBER> /findstationconfig searchdelay <NUMBER> Parameters: <GALAXYNAME> = name of current galaxy <GALAXYPATH> = full directory path for galaxy <MODE> = one of the available search modes player or galaxy <NUMBER> = any positive number or 0 INSTALLATION: Download ZIP for latest release from GitHub and extract it to <Avorion>\data\ directory, like with other mods. No vanilla script files will be overwritten, so there should be no problem with other mods or file changes due to game updates! Server/Client: The scripts are server- and client-side by now! Following files have to be available on the client for multiplayer games: scripts\entity\findstation\searchui.lua scripts\lib\findstation\common.lua scripts\lib\findstation\config.lua scripts\lib\findstation\sectorssearch.lua textures\icons\findstation\searchstation.png TIP: If you disable the /findstationui command on your server (and thus not using the search UI), no client files need to be installed at all! HOW TO: First use in galaxy: The mod tries to auto-detect the configuration when first search is executed in a galaxy. If the auto-configuration fails, you must execute /findstationconfig and configure the name of the galaxy manually (this has only to be done once per galaxy). /findstationconfig galaxy <GALAXYNAME> Example: /findstationconfig galaxy myfirstgalaxy If you want to use the search UI, you must enable it first by using: /findstationui If you want to hide/disable the UI and remove or uninstall the script, then use: /findstationui disable First use in galaxy (dedicated server with "--datapath"): The mod tries to auto-detect the configuration when first search is executed in a galaxy. If the auto-configuration fails, and you use the --datapath parameter for your server, you must execute /findstationconfig and configure the directory path of the galaxy (this has only to be done once per galaxy). Just use the same path as for "--datapath" plus the galaxy name. /findstationconfig galaxypath <GALAXYPATH> Example: /findstationconfig galaxypath C:\avorionserver\galaxies\myfirstgalaxy Search modes The available search modes define which sectors are searched for stations. searchmode: - defines the search mode to be used for all searches - possible values: - player (search only in sectors discovered by player) - galaxy (search in all sectors created in the galaxy) - default: player Performance tweaking If your searches are too slow or performance cost of searches is too high then you can modify some of the configs to tune the behavior (use /findstationconfig for this): framesectorloads: - defines the maximum number of searched/loaded sector files per frame (quite like "file reads per frame") - higher values mean faster search but more performance cost - possible values: 0 - 1000000 (0 disables the limit) - default: 10 maxresults: - defines after how many found results the search will stop - lower values means faster search in some cases, but also gives you less output obviously - possible values: 0 - 99999999 (0 disables the limit) - default: 30 Advanced server configuration These configs will help server admins to keep impact of searches on server load at a minumum and to prevent flood/spam! maxconcurrent: - defines the maximum number of concurrent searches, meaning how many players can have a search running at the same time - possible values: 0 - 99999999 (0 disables the limit) - default: 0 searchdelay: - defines the minimum time (in seconds) a player has to wait before he can start a new search, after each search - possible values: 0 - 99999999 (0 disables the limit) - default: 0 CHANGELOG & TODOs: Changelog: 0.6a - introduced search modes and related config (by default, the mod now searches only in sectors discovered by the player) - mod now tries to auto-detect its configuration for a galaxy (this may fail so you would have to configure manually with /findstationconfig) 0.5a - mod now supports linux servers *insert penguin here* - new configs maxconcurrent and searchdelay for anti-flood/-spam on servers (see "HOW TO - Advanced server configuration" section) - minor changes and bugfixes - decided to introduce proper versioning alpha4 - new command /findstationui to enable fancy new UI for station search (means menu item and window, check the screenshots) - the /findstationconfig command checks now if given galaxy or galaxypath is valid (valid means a "server.ini" file is found) - minor changes and bugfixes - additional mod structure and code refactoring (sry guys) alpha3 - implemented much faster way of checking for existing sectors (credit to LoSboccacc, screw the API!) - mod structure and code refactoring alpha2 - added per-frame processing of sector checks and station search alpha1 - initial experimental release TODOs: - split up workload of search into multiple frame-steps to minimize impact on performance - improve and optimize "sector picking"code for better general performance of searches - create nice dialog/window and menu item for the search - integration of anti-spam/-flood features and configs - linux support - (localization support for other game languages?) And now have fun searching those pesky hyperdrive-overloader parts ;D Feedback of any kind and bug-reports are of course welcome! Link to comment Share on other sites More sharing options...
LoSboccacc Posted March 6, 2017 Share Posted March 6, 2017 this is great! btw why do you check sector existence with the game api instead of iterating trough all the files that ends in v in the sector folder? function scandir(directory) local i, t, popen = 0, {}, io.popen local BinaryFormat = package.cpath:match("%p[\\|/]?%p(%a+)") if BinaryFormat == "dll" then --am bit rusty, might not be reliable when embedded, but you get the gist of it local cmd = 'dir "'..directory..'" /b /ad' else local cmd = 'ls -a "'..directory..'"' end local pfile = popen(cmd) for filename in pfile:lines() do if string.sub(filename,-1) == "v" then i = i + 1 t[i] = filename end end pfile:close() return t end Link to comment Share on other sites More sharing options...
w00zla Posted March 6, 2017 Author Share Posted March 6, 2017 this is great! btw why do you check sector existence with the game api instead of iterating trough all the files that ends in v in the sector folder? function scandir(directory) local i, t, popen = 0, {}, io.popen local BinaryFormat = package.cpath:match("%p[\\|/]?%p(%a+)") if BinaryFormat == "dll" then --am bit rusty, might not be reliable when embedded, but you get the gist of it local cmd = 'dir "'..directory..'" /b /ad' else local cmd = 'ls -a "'..directory..'"' end local pfile = popen(cmd) for filename in pfile:lines() do if string.sub(filename,-1) == "v" then i = i + 1 t[i] = filename end end pfile:close() return t end i also came up with this idea, but decided for the api variant for two reasons: the api calls should in theory be faster than file reads (we do a million(!) checks after all) because the engine providing the api should have some kind of cached data of the sectors in memory or the data/binary files, but that does not have to be the case actually. i also was not aware of a good function in LUA to scan directory files or check file existence without opening/reading the file, or installing some LUA modules and stuff. BUT your code looks very promising and I will try that of course, maybe we get even better performance then, what would be damn great :D Link to comment Share on other sites More sharing options...
LoSboccacc Posted March 6, 2017 Share Posted March 6, 2017 my guess is that a tiny tiny fraction of all those sectors are ever explored but of course only real life experiments can tell :) gonna check out the code in the evening if it works on my server, thanks again, it's a life saver Link to comment Share on other sites More sharing options...
w00zla Posted March 6, 2017 Author Share Posted March 6, 2017 with little modifications to your code i managed implement the directory scan nicely and now the check for existing sectors is so much faster and takes like no performance hit ;D thats just perfect! and a bit sad that game API calls are so slow in comparison... updated the OP and released version alpha3: added super fast sector checking routine, now searches finish in few seconds and only have little performance cost refactored mod files and code (please delete all old mod files if you are upgrading!) Link to comment Share on other sites More sharing options...
Dharaos Posted March 7, 2017 Share Posted March 7, 2017 It works! My eyes can rest a bit easier now, thank you. :D Link to comment Share on other sites More sharing options...
LoSboccacc Posted March 7, 2017 Share Posted March 7, 2017 with little modifications to your code i managed implement the directory scan nicely and now the check for existing sectors is so much faster and takes like no performance hit ;D thats just perfect! and a bit sad that game API calls are so slow in comparison... updated the OP and released version alpha3: added super fast sector checking routine, now searches finish in few seconds and only have little performance cost refactored mod files and code (please delete all old mod files if you are upgrading!) <3 Link to comment Share on other sites More sharing options...
Devious Posted March 7, 2017 Share Posted March 7, 2017 Nice to see that you are making progress on the mod :) Would it be viable to test this on a busy and heavily modded server yet? Our server is sometimes near 100% server load so I'm curious towards the performance if there is not much headroom left for the search. Link to comment Share on other sites More sharing options...
w00zla Posted March 7, 2017 Author Share Posted March 7, 2017 Nice to see that you are making progress on the mod :) Would it be viable to test this on a busy and heavily modded server yet? Our server is sometimes near 100% server load so I'm curious towards the performance if there is not much headroom left for the search. I think you can already start testing it! the new method of sector-checking is really fast and has no performance cost at all, and you can configure how many sectors are loaded by the server per-frame for each search (see "Performance Tweaking" in the HOW TO section of op). Give the config framesectorloads a value of i.e. 1 and the server will only load and search one sector per frame which will make the searches pretty slow, but should have like no impact on performance (depends a bit on disks and iops of the server). BUT: the described limit only applies per search and per player atm! A single player cannot execute multiple searches in parallel, meaning only one at a time, but multiple players could spam the findstation command and thus produce (heavy) load on the server independently from the configs. On the other side, when players dont abuse stuff i am pretty sure the searches do well ;) EDIT: decided to add the integration of anti-spam/-flood features and configs to the TODOs Link to comment Share on other sites More sharing options...
w00zla Posted March 9, 2017 Author Share Posted March 9, 2017 updated the OP and released version alpha4: - new command /findstationui to enable fancy new UI for station search - the /findstationconfig command checks now if given galaxy or galaxypath is valid (valid means a "server.ini" file is found) - minor changes and bugfixes - additional mod structure and code refactoring (sry guys, but please delete old mod-files again!) Preview of the new UI: Link to comment Share on other sites More sharing options...
LoSboccacc Posted March 10, 2017 Share Posted March 10, 2017 nice! now it only needs to get tied to the merchant dialogue after 'where's your home sector', so that it feels part of the game itself. bonus point if each station has a chance of not being known based off distance. Link to comment Share on other sites More sharing options...
w00zla Posted March 10, 2017 Author Share Posted March 10, 2017 nice! now it only needs to get tied to the merchant dialogue after 'where's your home sector', so that it feels part of the game itself. bonus point if each station has a chance of not being known based off distance. dont really know what you mean by integrating it into the merchant dialogues, as the mod is intended to provide galaxy-wide search for all sectors. same goes for your second point. but i already thought of creating another mod which could be a system upgrade (something like a "super scanner" ;D) which integrates the search, and has properties like i.e. maximum search distance, max results, power consumption... Link to comment Share on other sites More sharing options...
Andy-Blackknight Posted March 10, 2017 Share Posted March 10, 2017 its for stable and def version? Link to comment Share on other sites More sharing options...
w00zla Posted March 10, 2017 Author Share Posted March 10, 2017 its for stable and def version? yes, its tested with 0.10.2 stable and 0.10.5 dev/beta ! Link to comment Share on other sites More sharing options...
Andy-Blackknight Posted March 10, 2017 Share Posted March 10, 2017 You should be able to search for Player ships ok kannst ja deutsch hast du den mod gemacht? Link to comment Share on other sites More sharing options...
w00zla Posted March 10, 2017 Author Share Posted March 10, 2017 You should be able to search for Player ships ok kannst ja deutsch hast du den mod gemacht? unfortunatelly, the info about (player-)ships is stored in the binary sector files, and the mod can only read the XML files at the moment! maybe this will change sometime or when the game API for sector data is available... Link to comment Share on other sites More sharing options...
Pob Posted March 11, 2017 Share Posted March 11, 2017 Thank you for this. Link to comment Share on other sites More sharing options...
w00zla Posted March 12, 2017 Author Share Posted March 12, 2017 Good news for all the server admins ;D I updated the OP and released version 0.5alpha: - mod now supports linux servers! - new configs maxconcurrent and searchdelay for anti-flood/-spam on servers (see "HOW TO - Advanced server configuration" section in OP) - minor changes and bugfixes - decided to introduce proper versioning, as i did not plan to release this many updates for the mod ;) Link to comment Share on other sites More sharing options...
w00zla Posted March 12, 2017 Author Share Posted March 12, 2017 forgot to disable DEBUG output in the latest release :o please re-download if you have gotten the v0.5a ZIP before time of this posting, or change line "local debugoutput = true" to "local debugoutput = false" in file scripts/lib/findstation/common.lua ! Link to comment Share on other sites More sharing options...
Sny Posted March 13, 2017 Share Posted March 13, 2017 Any way to make it only show stations that a given player has discovered? Right now it seems like it reveals areas a player has never been to (but obviously another player has) Link to comment Share on other sites More sharing options...
w00zla Posted March 14, 2017 Author Share Posted March 14, 2017 Any way to make it only show stations that a given player has discovered? Right now it seems like it reveals areas a player has never been to (but obviously another player has) this is under investigation at the moment 8) the info about discovered, and not created sectors, is stored in the player.dat files, which are binary and their structure is pretty unknown (I didnt find any usable info at least). If I can reverse engineer the file structure and parse its data in an reasonable amount of time/performance, this feature may become available with the next update. But i cannot promise and maybe this will only be available with the release of proper API access to sector/galaxymap data! Link to comment Share on other sites More sharing options...
PolishRenegade Posted March 16, 2017 Share Posted March 16, 2017 This is great! Thank you! Already saved me hours of useless searching and pages on my notepad. One small caveat, it appears it searches in discovered but not explored (never entered) systems as well. Also wondering if its possible to show the level of some of the stations? (e.g. Turret Factories) Cheers Link to comment Share on other sites More sharing options...
hurrrr Posted March 17, 2017 Share Posted March 17, 2017 Would this work installed only on my client? The server owner is anti-modding. Link to comment Share on other sites More sharing options...
Thundercraft Posted March 18, 2017 Share Posted March 18, 2017 ...added super fast sector checking routine, now searches finish in few seconds and only have little performance cost... Could you elaborate on "only have little performance cost"? Will this mod definitely have at least a minor impact on performance? Is it easily noticeable? Or only barely noticeable? Also: Would the performance hit be even more noticeable on a server that regularly has 10 to 24 players at any given time? This was at least considered on the multiplayer server where I play. However, even after upgrading the server, there are lag issues. And the concern is that - because this is server-side - this mod might make existing lag/performance issues worse. (The thinking is that there is not enough "headroom", performance wise, for something like this.) Link to comment Share on other sites More sharing options...
Taurec Posted March 18, 2017 Share Posted March 18, 2017 Hmmm... I have aproblem: When i open chat and give the command "/findstationconfig galaxy Utopia" i become the errormessage: Unknown command findstationui. Utopia is the name of my galaxy. Or must i type the command in a other console than chat? Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now