What to do when things break

If you create a new branch, add a submodule there, and then switch back to a branch without that submodule, you still have the submodule directory as an untracked directory

$ git checkout -b add-library
Switched to a new branch 'add-library'

$ git submodule add <library>
Cloning into 'library'...
...

$ git commit -am 'adding library library'
[add-library 4445836] adding library library
 2 files changed, 4 insertions(+)
 create mode 160000 library

$ git checkout master
warning: unable to rmdir library: Directory not empty
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	library/

nothing added to commit but untracked files present (use "git add" to track)

Fix

$ git clean -fdx
Removing library/

$ git checkout add-library
Switched to branch 'add-library'

$ ls library/

$ git submodule update --init
Submodule path 'library': checked out 'b8dda6aa182ea4464f3f3264b11e0268545172af'

$ ls library/
Makefile		src