Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


davidcole

6018 posts

Uber Geek

Trusted

#251252 15-Jun-2019 08:27
Send private message

I have a number of docker hosts - for arguments sake lets use 3 as an example. On those 3 hosts I have a total of 10 applications. Each will be standalone, or part of a stack. Most of the applications do hold some persistent data (configuration files and the like).

 

I have in the past, used a flat svn structure.

 

  • docker-data

     

    • stack1

       

      • app1
      • app2
      • app3
    • stack2

       

      • app4
      • app5
    • stack3

       

      • app6
      • app7
    • app8
    • app9

The whole structure and the config files would be under svn and loaded on all 3 hosts. Having to remember which host had which apps, and make changes and commit as needed.

 

Where host 1 = stack 1, host 2 = stack2, host3 = stack3 and app8 and 9

 

As part of rebuilding I was going to look at moving to git and a better structure.

 

I did see someone suggest the my entire structure go into git as the master
With a branch for each host.

 

  • master

     

    • docker-data

       

      • stack1

         

        • app1
        • app2
        • app3
      • stack2

         

        • app4
        • app5
      • stack3

         

        • app6
        • app7
      • app8
      • app9
  • host1

     

    • docker-data

       

      • stack1

         

        • app1
        • app2
        • app3
  •  

This seemed quite a good approach. But what struck me is how to set it up initially.

 

If all my apps are in the master how do I initialise the branch for host one and only pull app1, 2 and 3?

 

I assume I them do a merge back to master if any of those configurations change.

 

And finally, I want to move app8 into host1's branch from host 3.

 

Is this over complicated? and are there any posts, or commands that would help me facilitate this?

 

 





Previously known as psycik

Home Assistant: Gigabyte AMD A8 Brix, Home Assistant with Aeotech ZWave Controller, Raspberry PI, Wemos D1 Mini, Zwave, Shelly Humidity and Temperature sensors
Media:Chromecast v2, ATV4 4k, ATV4, HDHomeRun Dual
Server
Host Plex Server 3x3TB, 4x4TB using MergerFS, Samsung 850 evo 512 GB SSD, Proxmox Server with 1xW10, 2xUbuntu 22.04 LTS, Backblaze Backups, usenetprime.com fastmail.com Sharesies Trakt.TV Sharesight 


Create new topic
marpada
473 posts

Ultimate Geek


  #2258683 15-Jun-2019 11:13
Send private message

Why not use k8s or any other scheduler and let the cluster manage where the containers are deployed?


 
 
 

Trade NZ and US shares and funds with Sharesies (affiliate link).
davidcole

6018 posts

Uber Geek

Trusted

  #2258696 15-Jun-2019 11:59
Send private message

marpada:

Why not use k8s or any other scheduler and let the cluster manage where the containers are deployed?



But don’t you still have to manage all the configurations of the apps, so I haven’t really gained anything




Previously known as psycik

Home Assistant: Gigabyte AMD A8 Brix, Home Assistant with Aeotech ZWave Controller, Raspberry PI, Wemos D1 Mini, Zwave, Shelly Humidity and Temperature sensors
Media:Chromecast v2, ATV4 4k, ATV4, HDHomeRun Dual
Server
Host Plex Server 3x3TB, 4x4TB using MergerFS, Samsung 850 evo 512 GB SSD, Proxmox Server with 1xW10, 2xUbuntu 22.04 LTS, Backblaze Backups, usenetprime.com fastmail.com Sharesies Trakt.TV Sharesight 


davidcole

6018 posts

Uber Geek

Trusted

  #2258735 15-Jun-2019 13:49
Send private message

So I had a play around, as with in a branch, if I removed any folders in the branch, them obviously it counted that as a change.

 

 

 

But what I found you can do, it create your branch of the master, and switch to it (git checkout -b new-system_branch)

 

The using sparse-checkout, you can choose the folders to display.  It's downloaded everything, so you're not doing it to save space, but it only shows you want you want to see.   So you can stick with the flat structure, but then only list the folders you want.

 

So on host1, you'd do:

 

 

 

git config core.sparsecheckout true

 

echo stack1/ > .git/info/sparse-checkout

 

git read-tree -m -u HEAD

 

On host 3

 

git config core.sparsecheckout true

 

echo stack3/ > .git/info/sparse-checkout

 

echo app8/ > .git/info/sparse-checkout

 

echo app9/ > .git/info/sparse-checkout

 

git read-tree -m -u HEAD

 

 

 

Then after any changes, you can send them back to the master.  The only consideration would be changes to say app9 on host3, if you then wanted to move app9 to host1, you would have to merge from master to host1s branch to get any changes made.

 

 





Previously known as psycik

Home Assistant: Gigabyte AMD A8 Brix, Home Assistant with Aeotech ZWave Controller, Raspberry PI, Wemos D1 Mini, Zwave, Shelly Humidity and Temperature sensors
Media:Chromecast v2, ATV4 4k, ATV4, HDHomeRun Dual
Server
Host Plex Server 3x3TB, 4x4TB using MergerFS, Samsung 850 evo 512 GB SSD, Proxmox Server with 1xW10, 2xUbuntu 22.04 LTS, Backblaze Backups, usenetprime.com fastmail.com Sharesies Trakt.TV Sharesight 




marpada
473 posts

Ultimate Geek


  #2258842 15-Jun-2019 18:38
Send private message

davidcole:
marpada:

 

Why not use k8s or any other scheduler and let the cluster manage where the containers are deployed?

 



But don’t you still have to manage all the configurations of the apps, so I haven’t really gained anything

 

Developers usually don't couple the environment details of where/how the app is with the app, but include the docker-compose.yml or k8s configuration, that way they can easily test locally and deploy it to multiple environments (QA, staging, prod). The CI/CD system builds the containers and deploys then to the desired cluster . The app repo might include the configuration of the CI/CD jobs, but it's decoupled so the project can still be deployed with any other system.

 

Traditionally each app has its own repo and project CD/CI project, so it's easy to deploy apps individually by just pushing a commit to the right branch.

 

Of course there are many ways of doing it and yours it's perfectly valid it it floats your boat, it depends of what are you trying to achieve and the different trade-offs.

 

 

 

 


nzkc
1556 posts

Uber Geek


  #2258848 15-Jun-2019 18:57
Send private message

I'm with @marpada on this one.  I'd keep the configuration of the applications separate from the deployment configuration.

 

How are you deploying things?  If its something like Ansible then I'd put where apps run with that. Very likely in its own repository.  If you're manually deploying it, by hand, I wouldnt even bother!  In fact I dont :)  I just use a docker run command and save that as a script somewhere.  Then run it on machines I want it running on.  That script can, quite frankly, go anywhere.

 

With regards to your directory structure above, I'm not certain I follow it exactly or even if it a directory structure.... if it is - I dont think using git branches is right.  Perhaps look at submodules - though thats not quite right either.

 

I did immediately think, why not use symbolic links (assuming Linux or similar of course).  However; there's lots of ifs/buts/maybes around that.  And Im guessing you'd ask something like "where do I keep which hosts get which symbolic links".  Well that could be a file you run that looks up the host (by name or something else) and then creates the links appropriately.  Or you have a separate file per host and call that.  All of that could go alongside your docker configuration if you wanted.  Although I'd be tempted to look at a separate repository for that.


Create new topic





News and reviews »

Bolt Launches in New Zealand
Posted 11-Jun-2025 00:00


Suunto Run Review
Posted 10-Jun-2025 10:44


Freeview Satellite TV Brings HD Viewing to More New Zealanders
Posted 5-Jun-2025 11:50


HP OmniBook Ultra Flip 14-inch Review
Posted 3-Jun-2025 14:40


Flip Phones Are Back as HMD Reimagines an Iconic Style
Posted 30-May-2025 17:06


Hundreds of School Students Receive Laptops Through Spark Partnership With Quadrent's Green Lease
Posted 30-May-2025 16:57


AI Report Reveals Trust Is Key to Unlocking Its Potential in Aotearoa
Posted 30-May-2025 16:55


Galaxy Tab S10 FE Series Brings Intelligent Experiences to the Forefront with Premium, Versatile Design
Posted 30-May-2025 16:14


New OPPO Watch X2 Launches in New Zealand
Posted 29-May-2025 16:08


Synology Premiers a New Lineup of Advanced Data Management Solutions
Posted 29-May-2025 16:04


Dyson Launches Its Slimmest Vaccum Cleaner PencilVac
Posted 29-May-2025 15:50


OPPO Reno13 Pro 5G Review 
Posted 29-May-2025 15:33


Logitech Introduces New G522 Gaming Headset
Posted 21-May-2025 19:01


LG Announces New Ultragear OLED Range for 2025
Posted 20-May-2025 16:35


Sandisk Raises the Bar With WD_BLACK SN8100 NVME SSD
Posted 20-May-2025 16:29









Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.







GoodSync is the easiest file sync and backup for Windows and Mac