After sleeping on it, I’ve just tried this:
It’s not enough to set the icons to embedded, as mentioned above. You also need to add LogicalName tags like this on the .csproj file:
<EmbeddedResource Include="Icon.bmp">
<LogicalName>Icon.bmp</LogicalName>
</EmbeddedResource>
Same thing for the Icon.ico, although that was working previously, but added it anyway.
Now I have my custom icon on the window (form) bar and on the task bar.