[Discussion] Mod Loader

[infal]

Is anyone else working on any kind of mod loader? (is the dev team working on a mod loader?!)

 

I've been tinkering with a simple system that would require server.lua to be modified but no other files, making it easy to install.

 

Mod loader scans subdirectories of data/scripts/mods/ for files named modinfo.lua, if it finds a file and the file returns a modinfo table, it loads the mod.

 

The files structure I have is like

 

[pre]

scripts/

modloader/

.git/

api/

lib/

modloader.lua

readme.md

mods/

cargoui/

.git/

modinfo.lua

interface.lua

playerscript.lua

config.lua

readme.md

infinion/

.git/

modinfo.lua

playerscript.lua

config.lua

readme.md

[/pre]

 

 

Functions:

registerAdminUIModule

 

 

Overview: Each time I made a mod I was also making an admin ui so that it was easy to cheat in items etc for testing. Each admin ui script had to be added to the player ship by /run command, which was tedious so I decided to make a unified Mod Loader Admin UI which can just load modules and give them a UI Container object to render in to.

 

First implemented: v1.2.0

 

[pre]

-- Register an Admin UI module that will be available to server admin and singleplayer

-- @param modName The name of the mod registering the ui panel. This is the string that will display in the listbox

-- @param script The script to add to the ship eg "data/scripts/mods/mymod/myscript.lua"

-- @return 0 if the function call failed, 1 for success

function registerAdminUIModule(modName, script)

[/pre]

 

 

 

registerPlayerShipScript

 

 

Overview: I found that a lot of mods I was writing needed a script to be attached to the player ship whenever they changed ships. It's really just a matter of registering for the onChangeShips callback but I was doing it often enough that it made sense to have a single function doing all the work rather than rewrite the event handling code over and over.

 

First implemented: v1.2.0

 

[pre]

-- Register a script for ModLoader to add to player ships automatically any time

-- the player enters a ship.

-- @param script The script to add to the ship eg "data/scripts/mods/mymod/myscript.lua"

-- @param removeOnExit [true/false] Should the script be removed from the ship when the player exits the ship?

-- @param allowDrone [true/false] Should the script be added to the drone?

-- @param adminOnly [true/false] Is this script only for admins?

function registerPlayerShipScript(script, removeOnExit, allowDrone, adminOnly)

[/pre]

 

 

registerSystemUpgradeAsLoot

 

 

Overview: This function makes it easy to add System Upgrade modules as loot to NPCs. This functionality is a little bit of work because you have to handle several events, onSectorEntered/Left and onEntityCreate and then there is some processing to find the right NPC. Having one single generic system to handle this made sense.

 

First implemented: v1.2.0

 

[pre]

-- Registers system upgrades as loot to be added to NPCs

--

-- @param script The filename for the system upgrade eg "data/scripts/mods/mymods/myupgrade.lua"

-- @param npc The type of NPC you want. Look in modloader/lib/enums.lua for values eg NPC.Xsotan

-- @param class The class of ship. eg ShipClass.Cruiser  or  ShipClass.Any etc

-- @param chance [1-100] The chance of this loot being added to an NPC

-- @return An integer id for the loot table incase you need to remove it at a certain point mid-session

function registerSystemUpgradeAsLoot(script, npc, class, distribution, chance)

[/pre]

 

 

 

 

Links:

Simple Mod Loader on GitHub -or- Zip file Download

The basic mod loader. I haven't included a modified server.lua file or instructions on what code to add etc as part of the git but they are in the zip file for download

 

Mod Template on GitHub -or- Zip file Download

Some files to help get a mod started.

 

Cargo UI mod on GitHub -or- Forum Topic

This mod shows how to have a script attach to the player's ship.

 

Infinion Corporation mod on Github -or- Forum Topic

This mod shows how to generate a custom station in random sectors and how to make the station easily accessible to other mods.

Infinion Irregular Goods Transportation Addon mod on Github

This mod shows how to add a script to the custom station above, add loot to pirates when the player enters a sector and how to change behaviour of military ships by removing a default script and adding a custom script in it's place.

 

Infinion Corporation Faction Modules -or- Forum Topic

This mod adds 3 system upgrade modules that are mostly just base game turret control systems but with boosted stats. It shows how to add them as loot drops to NPCs, very selectively, and how to make a module for the unified Admin UI.

[Wilponderoci]

i would love to see a user friendly fully functional version of a mod loader tool

[GhoStalker]

I was thinking of something like this as well. I want to be able to update the game, and not have to tracked down all the single changes that were made to customize the server.

 

I was trying to implement something like you laid out that was pulled from a git repo on restart so I didn't have to login to the linux box every time I wanted to update a mod.

 

+1

[rater193]

"modloader" is an ambiguous term, but it would be nice to be able to load mods from a general directory without needing to modify exsisting files or folders. Ill give a crack at it, at least until the developer makes something official. I will post it on the forums when i get it able to be stable

[rater193]

btw, is there any specific thing you are looking for in the mod loader? let me know so i am aware of it, and can add it in.

[w00zla]

i have the same opinion like rater193.

a general modloader would be great, but your approach requires too much tinkering by the end-user.

an approach with scanning special mod folders for scripts and including them automagically would be much more handy and less error-prone in my opinion! what do you guys think?

[infal]

I've only been scripting LUA for a week or so. I glanced briefly at LUA file system options and they seem clunky, at best.

 

Of course I'd prefer to have mods loaded without any file editing... but as a compromise, I'd rather edit 1 line per mod in a single file than randomly hacking through or overwriting base game files, which is the current standard.

 

Ideally, load mods from zip files in data/mods/ directory.

 

I'm using onPlayerLogIn event from server.lua to attach a script specified by mod info file to the player and/or run an onInitialize function in the mod info file. That allows a lot of functionality because the events system is quite powerful. I don't think a mod loader needs to do "all the things", it just needs to load the mod and let the mod author have access to the events system.

 

If zip files aren't possible then iterate through a directory for files like the mod info above and load it in.

 

Trade goods and production recipes are a challenge. I'm not sure if there's a way to make changes to the goods table and production table. The problem is that there is no global instance.

[w00zla]

I've only been scripting LUA for a week or so. I glanced briefly at LUA file system options and they seem clunky, at best.

 

Of course I'd prefer to have mods loaded without any file editing... but as a compromise, I'd rather edit 1 line per mod in a single file than randomly hacking through or overwriting base game files, which is the current standard.

 

Ideally, load mods from zip files in data/mods/ directory.

 

I'm using onPlayerLogIn event from server.lua to attach a script specified by mod info file to the player and/or run an onInitialize function in the mod info file. That allows a lot of functionality because the events system is quite powerful. I don't think a mod loader needs to do "all the things", it just needs to load the mod and let the mod author have access to the events system.

 

If zip files aren't possible then iterate through a directory for files like the mod info above and load it in.

 

Trade goods and production recipes are a challenge. I'm not sure if there's a way to make changes to the goods table and production table. The problem is that there is no global instance.

 

Dont take my comment negatively, i like your general approach and think your code is already a really good starting point :)

 

I dont have much LUA experience for myself (has been several years since last LUA script before Avorion) and was also struggling with the filesystem/io capabilities of LUA! But luckily LoSboccacc came up with a good solution for scanning directories for files, and i made working function out of it for my findstation mod (works on win and linux):

 

require("stringutility")

function scandir(directory, pattern)

    local i, t, popen = 0, {}, io.popen
local BinaryFormat = string.sub(package.cpath,-3)
local cmd = ""
if not string.ends(directory, "/") then
	directory = directory .. "/"				
end
local path = directory	
if pattern then 	
	path = path .. pattern
end
    if BinaryFormat == "dll" then
	path = string.gsub(path, "/", "\\")
	cmd =   'dir "'..path..'" /b /a-d'
    else
	path = string.gsub(path, "\\", "/")
	cmd = "ls " .. path
    end
    local pfile = popen(cmd)
    for filename in pfile:lines() do
	i = i + 1
	if string.starts(filename, directory) then
		t[i] = string.sub(filename, string.len(directory) + 1)
	else
		t[i] = filename
	end		
    end
    pfile:close()
    return t

end

 

this code gives you a table of all found files, with optional search pattern. use it like scandir("<Avorion>/data/mods/", "*.lua") to get all scripts in a dir 8)

 

for the possibilities of a script loader in general (this is just to my knowledge at the moment, please post if you have more info):

the only real hook to get custom scripts working without modifying any vanilla files is to create commands = create script files in the /data/scripts/coommands directory ... but we cant influence that in any way and command scripts will always have to go there and be on-demand by the user, so nothing of interest for the loader.

I also see only a few possibilities of hooks into scripting by modifying few basic vanilla scripts like "server.lua":

- events/callbacks of Server()

- events/callbacks of Sector()

- events/callbacks of Player()

then, of course, the direct attachment of scripts to the Player() entity.

If something like a basic entity-script loader is integrated, we could also have the "Entity():addScriptOnce()" mechanism provided directly to mods by the loader, and then have also:

- events/callbacks of Entity()

 

All other modifications of stuff, like i.e. trade goods and production recipes, spawns, loot, like you said, require the direct modification of the vanilla scripts, and this is not manageable by a loader without overwriting each "addScript()" line in every vanilla script :o

but the loader could of course modify "vanilla scripts of high interest and use" and provide some kind of replacement mechanism and/or simple API for i.e. adding new upgrades or goods, until workshop support comes its way ;)

 

[infal]

Thanks, that file io code works perfectly. I've integrated it to what I already had and am now loading mods without having to list them all in a file.

Is it okay for me to keep this code in the project and release it? I've got it in a seperate file (mods/modloader/lib/scandir.lua) so can add any credit etc you or LoSboccacc would request.

 

Some of the things you mention I have working through events; spawning entities, custom loot etc

eg

I have one mod that will create custom stations in certain sectors. That's pretty much all it does.

 

Then I have another mod which will add a dialog script to the custom stations to let the player purchase a dangerous goods permit. This second mod also makes pirates drop a cargo shielding system upgrade to prevent military ships being able to scan your cargo. To make this work I have to scan for military ships and replace their antismuggle script with a custom script.

[infal]

So I've been working on my version a little more.

 

I changed the file/folder structure so that mod loader and each mod are expected to be in their own directory. Looking at GhoStalker's case of wanting to use git, it made sense to contain everything in it's own directory.

 

I also changed it so that mods must provide a modinfo.lua as the entry point.

 

Still have to edit the server.lua to get the whole thing working. I haven't looked at any other options for that.

[w00zla]

From my side, no need to credit for the function, just happy another modder finds it useful ;D

 

Your progress and changes sound good so far, and I would really like to take a grasp on it!

did you forget to attach/link your new version or didnt you release it yet?

[infal]

Okay thanks! I wanted to wait to see if it was okay to include that code before I published it.

 

I've added links in the original post to the github projects for modloader and 3 mods that do different things showing that you can do quite a bit without needing to overwrite base files

[GhoStalker]

I've been super busy so I haven't had time to dig into this. I like the progress that's been made though. I am excited to see what comes out of this discussion. If you need some help testing it I would be more than willing to do so. I could adapt some of the scripts I have now to try to use your system.

[infal]

That'd be great, thanks.

 

I worked backwards a little putting this together; I had the mods mostly done and figured out what I needed a generic mod loading system to do to make those mods work. If you're able to adapt some of your own scripts, it may raise some new ideas on what needs to be included in a mod loader system.

 

If you happen to be testing on Linux, let me know if the scandir function is okay. I had to modify it to return files or directories but have no way to test the Linux code.

[infal]

Over the weekend I've been tinkering with a function that will make it easy for modders to add custom system upgrades to NPC ships as loot with a single function call

 

eg:

 

local scriptCargoShield = "data/scripts/mods/cargoshield/cargoshield.lua"
local distribution =
{
[5] = 0.1,	-- legendary
[4] = 1,	-- exotic
[3] = 8,	-- exceptional
[2] = 16,	-- rare
[1] = 32,	-- uncommon
[0] = 128,	-- common
}
local chancePiratesDropCargoShield = 30

registerSystemUpgradeAsLoot(scriptCargoShield, NPC.Pirate, ShipClass.Any, distribution, chancePiratesDropCargoShield)


-- distribution can be nil and the script will just use a default table.
registerSystemUpgradeAsLoot(scriptCargoShield, NPC.Civilian, ShipClass.Miner, nil, 40)

 

Any time the player enters a sector or anytime ships are created (eg pirate attack, xsotan attack, faction battle), mod loader will try to add loot to the ship(s).

 

This will make it a lot easier to get a custom system upgrade module into the game without hacking base files.

 

It's not quite ready to release but getting close.

[w00zla]

Over the weekend I've been tinkering with a function that will make it easy for modders to add custom system upgrades to NPC ships as loot with a single function call

.....

This will make it a lot easier to get a custom system upgrade module into the game without hacking base files.

 

It's not quite ready to release but getting close.

 

That sounds really good :) I will give the loader a try and test as soon as this is implemented completely, as this is what I need for my next mod idea!

[douglasg14b]

Typically if I was to think of a mod loader, it would be able to:

 

-- Load files into their appropriate directories 
-- Manage file merges (mod inserts code into an existing file)
-- Handles and warns of conflicts during merges
-- Handles mod dependencies (Mod B requires Mod A...etc) [i would optional this until we have an official mod repository]
-- Can be used without installing 3rd party libraries/SDKs/frameworks [This may be unrealistic given cross-platform compatibility. I would personally use .NET Core with all necessary dependencies packed with the executable with separate builds per supported OS]

[infal]

I'd think of that as more a launcher, but maybe that's just semantics. Regardless, what you've described is well beyond the scope of what I'm doing at the moment, but I like the ideas.

 

If no one works on something like that, I might look at it in the future. I'd probably want to put all of the core functionality in a c static lib and  make it compatible with zip and patch files.

 

With a proper launcher, a lot of the stuff I'm doing at the moment could be simplified to just a set of utility functions.

 

Having that functionality built in to the game engine would be best though as that would make it possible to have per-save mod configurations and client downloading from server.

[w00zla]

I'd think of that as more a launcher, but maybe that's just semantics. Regardless, what you've described is well beyond the scope of what I'm doing at the moment, but I like the ideas.

 

If no one works on something like that, I might look at it in the future. I'd probably want to put all of the core functionality in a c static lib and  make it compatible with zip and patch files.

 

With a proper launcher, a lot of the stuff I'm doing at the moment could be simplified to just a set of utility functions.

 

Having that functionality built in to the game engine would be best though as that would make it possible to have per-save mod configurations and client downloading from server.

 

I think that may all come with Steam Workshop support for the game?! Here is the question when the support will be released and if its worth the time developing such an fully-fledged loader??!!

[infal]

I think that may all come with Steam Workshop support for the game?! Here is the question when the support will be released and if its worth the time developing such an fully-fledged loader??!!

Exactly. I don't want to spend a bunch of time writing something that is literally already being officially developed, but on the other hand Mojang promised a modding API for Minecraft and 5 years later we have "resource packs". :o

[w00zla]

Exactly. I don't want to spend a bunch of time writing something that is literally already being officially developed, but on the other hand Mojang promised a modding API for Minecraft and 5 years later we have "resource packs". :o

 

Yeah, thats also true, and in Mojangs case really not the "interface" you wished for as a modder...

hmm, i really get itchy developer fingers when reading and thinking about a fully-fledged standalone mod-loader for Avorion, but I really would be interested in when koonschi plans to release the Steam Workshop support (a platform Mojang never had for MC modding)!

since he also plans to finish the game completely by end of the year (means out of early access), I doubt any bigger mods integration project will pay off ....

 

whats your opinion guys??

[infal]

Okay, I've got v1.2.0 of my mod loader up. It adds 3 functions for mod authors: One that lets you register system upgrades as loot drop for NPCs. One that lets you register a script that will be attached to the player ship any time they enter a ship, and, one that lets you add a module to a mod loader admin panel.

 

More info in the OP.

 

I've also added a mod template and a new mod that just shows how to add some basic, modified system upgrades. (just copies of the existing turret systems with boosted stats.)

 

I also forked a 1.3.x dev branch on GitHub

[Loony Liberator]

Hey Infal, quick question, once I've installed the mod loader and cargo ui exactly as you have displayed the Icon for Admin ui doesn't show.

Is there a command I have to do to initialise the mod loader? if not, is not having the .git/ folder something that is crucial for it to run because

that's not included in the download. Oh and I'm playing single player trying to get this to run.

[infal]

Hey Infal, quick question, once I've installed the mod loader and cargo ui exactly as you have displayed the Icon for Admin ui doesn't show.

Is there a command I have to do to initialise the mod loader? if not, is not having the .git/ folder something that is crucial for it to run because

that's not included in the download. Oh and I'm playing single player trying to get this to run.

Admin UI will only show if a mod registers an admin ui module AND the modloader/config.lua : t.giveAdminPanelAutomatically is set to true (which it is by default). If no mods call registerAdminUIModule, then the script isn't attached to the player ships, to save system resources. No sense having a script attached to every player ship if it's not doing anything meaningful.

 

Cargo UI doesn't actually add an admin ui module, as in my mods I'm using the admin ui to cheat in items for testing etc and cargo ui doesn't add any new items. The Cargo UI icon will only show when you're in a ship, not the drone.

 

If you want to manually attach the admin ui to a ship, you can use the command

/run Entity():addScript("modloader/lib/adminui.lua")

[infal]

Also, ModLoader puts a lot of info into the log files that you can look for to see if it's running, or if there was an error that gets the script set to invalid by the game engine.

 

Client log:

%appdata%/Avorion/clientlog *.txt

 

<ModLoader> Initializing ModLoader

<ModLoader> Registering mod: Cargo UI

 

...

 

<ModLoader> Running onInitialize for mod:Cargo UI

<ModLoader> Registering player ship script: mods/cargoui/interface.lua

 

 

Server log

%appdata%/Avorion/galaxies/savename/serverlog *.txt

 

<ModLoader> Running onInitialize for mod:Infinion Irregular Goods Transportation Addon

<ModLoader> Registering player ship script: data/scripts/modloader/lib/adminui.lua