As I've previously written about, I am working on a Video Synthesizer plugin for VCVRack. In this post I am going to explore the Output Sources diagram, the initial ideas behind it and how I am planning on changing them.
The Output Sources diagram is comprised of Shaders which I will be refering to in here as Filters and Media Viewers such as the PlainViewer.
The Filters are at the core of this project. The reason why they are called Shaders in the Diagram is because it is supposed to be an abstract Shader module to be used in all types Filter. A simple example is a Tint filter, that modifies the RGB colors of an image. In the diagram, a Filter has filter specific controls that can be used to modify the image effects after processing. In the background, these values would passed into an OpenGl shader program that processes them and in turn modifies the texture which is then rendered on the preview on the left preview side.
Similar to how input sources work in my previous post about Input Sources, the Filter would make the corresponding float value to the texture unsigned int available as output on the "REF" output. This could in return be used by other Filters and or output sources. As you can also notice, there is an "SRC" input, this is a Filter specific input that reads in an float that correspond to to a OpenGL texture. This way the Filter can take in textures from various components (Input Sources or other Filters) that it can then modify it and send it along.
The Filter component is a little bit like an input source however, because of it's extra functionalities I'll keep it in it's own category. There is a lot more work that needs to be done in terms of Shaders/Filters but as things stand right now, this is the general idea behind them. A problem that will eventually need to be addressed is how to get people to write custom filters to be used in the project. But this is a question for another time.
What's the point of a Video Synthesizer without nothing to see? Well this what the Media Viewers are for. They are responsible for presenting the final result from the input sources and filtering process. The PlainViewer shown in the diagram takes in an input "SRC" which is a float corresponding to a texture and it then displays the texture in the view rectangle to the right. It can take input from an Input Source or a Filter. Similarly it can be daisy chained and used as an Input source by other Media Viewers or Filters. This is great because it leaves room for more filtering combinations.
The plan with Media Viewers is to eventually expand it to handle various ways of displaying the proccessed textures. One such use case is a Viewer that streams the textures over the net so that it can be used for a performance or just any other usage.