check the installation directory and make sure you have libnvtt.dylib. On Mac that should be /Library/Frameworks/MonoGame.framework/Current/Tools.
You can use otool -L to check the dependencies and make sure nothing is missing. I got
/Library/Frameworks/MonoGame.framework/Current/Tools/libnvtt.dylib:
/Users/dean/Documents/Sandbox/dellis1972/nvidia-texture-tools/build-release/src/nvtt/libnvtt.dylib (compatibility version 0.0.0, current version 0.0.0)
/Users/dean/Documents/Sandbox/dellis1972/nvidia-texture-tools/build-release/src/nvimage/libnvimage.dylib (compatibility version 0.0.0, current version 0.0.0)
/Users/dean/Documents/Sandbox/dellis1972/nvidia-texture-tools/build-release/src/nvmath/libnvmath.dylib (compatibility version 0.0.0, current version 0.0.0)
/Users/dean/Documents/Sandbox/dellis1972/nvidia-texture-tools/build-release/src/nvcore/libnvcore.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.5.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.50.2)