Does the extension contain the new 3.8 templates? Yes the NuGet packages alone are enough. The “preview” part is that they’re still being debugged, but I don’t think the final release will have anything other than NuGet packages, templates, and online docs.
Interesting that you don’t see references. Can you share the contents of your csproj?
If it can’t find the namespace, that usually means it couldn’t restore NuGet packages. Try right-clicking on the solution node in the solution explorer and force it to restore. Also check the Output tab and select Package Manager from the dropdown to see if there are any errors.