So, to close this, the answer to the above question is yes, app icons will respect your preferred color scheme, but no, app icons won't update dynamically when you change your color scheme. Instead, they will keep their initial dark or light mode look from whatever you had your system set to at install time.
You can test it for yourself by installing the app embedded below (launch it in its own window).
Update: Alexey Rodionov has let me know that this even works for app shortcut icons. You can play with Alexey's demo or check out the screenshot below.
(Credits: The app is a remix of Alan Cutter's app fir-skirt.glitch.me.)