Strategies for holding multiple machine docker configurations.

Forums › Self-hosted › Strategies for holding multiple machine docker configurations.

davidcole

6018 posts

Uber Geek

Trusted

#251252 15-Jun-2019 08:27

I have a number of docker hosts - for arguments sake lets use 3 as an example. On those 3 hosts I have a total of 10 applications. Each will be standalone, or part of a stack. Most of the applications do hold some persistent data (configuration files and the like).

I have in the past, used a flat svn structure.

docker-data
- stack1
  - app1
  - app2
  - app3
- stack2
  - app4
  - app5
- stack3
  - app6
  - app7
- app8
- app9

The whole structure and the config files would be under svn and loaded on all 3 hosts. Having to remember which host had which apps, and make changes and commit as needed.

Where host 1 = stack 1, host 2 = stack2, host3 = stack3 and app8 and 9

As part of rebuilding I was going to look at moving to git and a better structure.

I did see someone suggest the my entire structure go into git as the master
With a branch for each host.

master
- docker-data
  - stack1
    - app1
    - app2
    - app3
  - stack2
    - app4
    - app5
  - stack3
    - app6
    - app7
  - app8
  - app9
host1
- docker-data
  - stack1
    - app1
    - app2
    - app3

This seemed quite a good approach. But what struck me is how to set it up initially.

If all my apps are in the master how do I initialise the branch for host one and only pull app1, 2 and 3?

I assume I them do a merge back to master if any of those configurations change.

And finally, I want to move app8 into host1's branch from host 3.

Is this over complicated? and are there any posts, or commands that would help me facilitate this?

Previously known as psycik

Home Assistant: Gigabyte AMD A8 Brix, Home Assistant with Aeotech ZWave Controller, Raspberry PI, Wemos D1 Mini, Zwave, Shelly Humidity and Temperature sensors
Media:Chromecast v2, ATV4 4k, ATV4, HDHomeRun Dual
Server Host Plex Server 3x3TB, 4x4TB using MergerFS, Samsung 850 evo 512 GB SSD, Proxmox Server with 1xW10, 2xUbuntu 22.04 LTS, Backblaze Backups, usenetprime.com fastmail.com Sharesies Trakt.TV Sharesight

marpada

473 posts

Ultimate Geek

#2258683 15-Jun-2019 11:13

Why not use k8s or any other scheduler and let the cluster manage where the containers are deployed?

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

davidcole

6018 posts

Uber Geek

Trusted

#2258696 15-Jun-2019 11:59

marpada:
Why not use k8s or any other scheduler and let the cluster manage where the containers are deployed?

But don’t you still have to manage all the configurations of the apps, so I haven’t really gained anything

davidcole

6018 posts

Uber Geek

Trusted

#2258735 15-Jun-2019 13:49

So I had a play around, as with in a branch, if I removed any folders in the branch, them obviously it counted that as a change.

But what I found you can do, it create your branch of the master, and switch to it (git checkout -b new-system_branch)

The using sparse-checkout, you can choose the folders to display. It's downloaded everything, so you're not doing it to save space, but it only shows you want you want to see. So you can stick with the flat structure, but then only list the folders you want.

So on host1, you'd do:

git config core.sparsecheckout true

echo stack1/ > .git/info/sparse-checkout

git read-tree -m -u HEAD

On host 3

git config core.sparsecheckout true

echo stack3/ > .git/info/sparse-checkout

echo app8/ > .git/info/sparse-checkout

echo app9/ > .git/info/sparse-checkout

git read-tree -m -u HEAD

Then after any changes, you can send them back to the master. The only consideration would be changes to say app9 on host3, if you then wanted to move app9 to host1, you would have to merge from master to host1s branch to get any changes made.

marpada

473 posts

Ultimate Geek

#2258842 15-Jun-2019 18:38

davidcole:
marpada:

Why not use k8s or any other scheduler and let the cluster manage where the containers are deployed?

But don’t you still have to manage all the configurations of the apps, so I haven’t really gained anything

Developers usually don't couple the environment details of where/how the app is with the app, but include the docker-compose.yml or k8s configuration, that way they can easily test locally and deploy it to multiple environments (QA, staging, prod). The CI/CD system builds the containers and deploys then to the desired cluster . The app repo might include the configuration of the CI/CD jobs, but it's decoupled so the project can still be deployed with any other system.

Traditionally each app has its own repo and project CD/CI project, so it's easy to deploy apps individually by just pushing a commit to the right branch.

Of course there are many ways of doing it and yours it's perfectly valid it it floats your boat, it depends of what are you trying to achieve and the different trade-offs.

nzkc

1556 posts

Uber Geek

#2258848 15-Jun-2019 18:57

I'm with @marpada on this one. I'd keep the configuration of the applications separate from the deployment configuration.

How are you deploying things? If its something like Ansible then I'd put where apps run with that. Very likely in its own repository. If you're manually deploying it, by hand, I wouldnt even bother! In fact I dont :) I just use a docker run command and save that as a script somewhere. Then run it on machines I want it running on. That script can, quite frankly, go anywhere.

With regards to your directory structure above, I'm not certain I follow it exactly or even if it a directory structure.... if it is - I dont think using git branches is right. Perhaps look at submodules - though thats not quite right either.

I did immediately think, why not use symbolic links (assuming Linux or similar of course). However; there's lots of ifs/buts/maybes around that. And Im guessing you'd ask something like "where do I keep which hosts get which symbolic links". Well that could be a file you run that looks up the host (by name or something else) and then creates the links appropriately. Or you have a separate file per host and call that. All of that could go alongside your docker configuration if you wanted. Although I'd be tempted to look at a separate repository for that.