Table of Contents
If you notice that the ownership of some (maybe all) files have changed and you
need to enter your root password when editing a file, you might
have pulled changes to the remote repository from within a container. While running
git pull, git changes the owner of the files that you pull to the user that runs
that command. Within a container, that is the root user - so git operations
should be ran outside of the container.
To fix this situation, change the owner back to yourself outside of the container by running:
$ sudo chown <user>:<group> -R .
Most of the
paver commands require a settings flag. If omitted, the flag defaults to
devstack. If you run into issues running
paver commands in a docker container, you should append
devstack_docker flag. For example:
$ paver update_assets --settings=devstack_docker
make static within the ecommerce container you could get an error
Error: Error: EBUSY: resource busy or locked, rmdir '/edx/app/ecommerce/ecommerce/ecommerce/static/build/'
To fix this, remove the directory manually outside of the container and run the command again.
If you see the error
no space left on device, Docker has run
out of space in its Docker.qcow2 file.
Here is an example error while running
... 32d52c166025: Extracting [==================================================>] 1.598 GB/1.598 GB ERROR: failed to register layer: Error processing tar file(exit status 1): write /edx/app/edxapp/edx-platform/.git/objects/pack/pack-4ff9873be2ca8ab77d4b0b302249676a37b3cd4b.pack: no space left on device make: *** [pull] Error 1
Try this first to clean up dangling images:
docker system prune -f # (This is very safe, so try this first.)
If you are still seeing issues, you can try cleaning up dangling volumes.
Bring up all containers.
Remove all unused volumes. Warning: this will remove all Docker data on your system that is not currently in use by a container, which is why it’s important to run the previous step. Otherwise, this will wipe out your Devstack data.
docker volume prune -f
While provisioning, some have seen the following error:
... cwd = os.getcwdu() OSError: [Errno 2] No such file or directory make: *** [dev.provision.services] Error 1
This issue can be worked around, but there’s no guaranteed method to do so. Rebooting and restarting Docker does not seem to correct the issue. It may be an issue that is exacerbated by our use of sync (which typically speeds up the provisioning process on Mac), so you can try the following:
# repeat the following until you get past the error. make stop make dev.provision.<service>
Once you get past the issue, you should be able to continue to use sync versions of the make targets.
While provisioning, some have seen the following error:
... Build failed running pavelib.assets.update_assets: Subprocess return code: 137
This error is an indication that your docker process died during execution. Most likely, this error is due to running out of memory. Try increasing the memory allocated to Docker (Recommended: 8 CPUs, 10 GB Memory, 2GB Swap).
On the Mac, this often manifests as the
hyperkit process using a high
percentage of available CPU resources. To identify the container(s)
responsible for the CPU usage:
Once you’ve identified a container using too much CPU time, check its logs; for example:
The most common culprit is an infinite restart loop where an error during
service startup causes the process to exit, but we’ve configured
docker-compose to immediately try starting it again (so the container will
stay running long enough for you to use a shell to investigate and fix the
problem). Make sure the set of packages installed in the container matches
what your current code branch expects; you may need to rerun
pip on a
requirements file or pull new container images that already have the required
package versions installed.
When trying to check out a branch, you may see an error like this:
git checkout jj/REV-666-implement-evil-feature > error: pathspec 'jj/REV-666-implement-evil-feature' did not match any file(s) known to git
If you are sure you have (i) recently run
git fetch and (ii) didn’t misspell the
branch name, then it is possible your repository is set in “single-branch” mode, meaning
that it is configured to only fetch
master. Although devstack currently clones services’
repositories with all their branches, devstacks provisioned before September 2020
will start out with single-branch repositories. You check if your repository is in this
state by running
git branch -r. If you only see a couple of entries
origin/HEAD), then your local repository is in single-branch
You can manually reconfigure your repository to pull all branches by running these commands from within the repository:
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" git fetch origin git checkout jj/REV-666-implement-evil-feature > Switched to branch 'jj/REV-666-implement-evil-feature'.
git is powerful but complex; you may occasionally find your respository in a
confusing state. This problem isn’t devstack-specific.
If you find yourself stuck, folks in the edX-internal or Open edX Slack workspaces may be able to give you a hand.
Alternatively, if you are at a roadblock and don’t care about any changes you’ve made to your local copy of the repository (i.e., you have pushed or otherwise saved your work elsewhere) then you can always delete the repository and start over again:
rm -rf ./<repository> git clone firstname.lastname@example.org:edx/<repository>
Finally, if you regularly find yourself mystified by git, consider reading
through Understanding Git Conceptually. It explains core Git principles in way
that makes it easier to use the simpler
git commands more effectively
and easier to use the more complicated
git commands when you have to.
docker-sync is no longer actively supported
Check your version and make sure you are running 0.4.6 or above:
If not, upgrade to the latest version:
gem update docker-sync
If you are having issues with docker sync, try the following:
make stop docker-sync stop docker-sync clean
The performance improvements provided by cached consistency mode for volume mounts introduced in Docker CE Edge 17.04 are still not good enough. It’s possible that the “delegated” consistency mode will be enough to no longer need docker-sync, but this feature hasn’t been fully implemented yet (as of Docker 17.12.0-ce, “delegated” behaves the same as “cached”). There is a GitHub issue which explains the current status of implementing delegated consistency mode.