Synthography

Synthography[1] is the method of generating digital media synthetically using machine learning. This is distinct from other graphic creation and editing methods in that synthography uses artificial intelligence art text-to-image models to generate synthetic media. Synthography is the method used, not the output itself. It is commonly achieved by prompt engineering text descriptions as input to create or edit a desired image.[2][3] Where photography turns light into images, synthography turns ideas into images.[4]

A synthographic image created with a fine-tuned Stable Diffusion model and outpainting

Text-to-image models, algorithms, and software are tools used in synthography that are designed to have technical proficiency in creating the resulting artificial intelligence art output based on human input. Synthography typically uses text-to-image models to synthesize new images as a derivative of the training, validation, and test data sets on which the text-to-image models were trained. Another technique is to fine-tune an already trained model on a image dataset in order to teach it a specific concept.

Etymology

From Latin synthesis "collection, composition", from Greek synthesis "composition, a putting together".[5]
"-graphy" is the word-forming element meaning "process of writing or recording" or "a writing, recording, or description" (in modern use especially in forming names of descriptive sciences). From French or German -graphie, from Greek -graphia "description of," used in abstract nouns from graphein earlier "to draw, represent by lines drawn," originally "to scrape, scratch" (on clay tablets with a stylus).[6] The term is still in its infancy, as early adopters are using different terminology for this technique. Other names are prompt engineering, image synthesis and artificial intelligence art.

History

The event known to have started the broad usage of text-to-image models is the publication of DALL-E by OpenAI in January 2021.[7] While it was not released to the public, CLIP (Contrastive Language-Image Pre-training) was open-sourced, which led to a succesion of implementations with other generators such as Generative adversarial networks and Diffusion models.[8][9] The next big event, which led to a rise in popularity of such technique, was the release of DALL-E 2 in April 2022. After slowly releasing it as a private beta, it became public in July 2022. In August 2022, Stable Diffusion was open-sourced by Stability AI,[10] which fostered a community-led movement.

Methodology

As synthography refers to the method of generating AI visual imagery, these are the mediums or categories used in the method.

Synthography Mediums
input \ output text image 3D model video
text chatbot text-to-image text-to-3D text-to-video
image image-to-text image-to-image image-to-3D image-to-video
video video-to-video
Legend
white backgroundDoesn't exist yet
light green backgroundCurrently exists in academia or beta
green backgroundExists commercially or widely available

(Note: text-to-speech and speech-to-text are purposely omitted in the table since that can simply be performed by dictation/transcription software and therefore is implied by the 'text' row and column. Also note that the mediums listed are of the class of medium, not specific instances of it ie: chatbot instead of ChatGPT.)

References
  1. Reinhuber, Elke (2021-12-02). "Synthography–An Invitation to Reconsider the Rapidly Changing Toolkit of Digital Image Creation as a New Genre Beyond Photography". scholar.google.com. Springer, Cham. Retrieved 2022-12-20.
  2. Smith, Thomas (2022-10-26). "What is Synthography? An Interview With Mark Milstein - Synthetic Engineers". syntheticengineers.com. Synthetic Engineers. Retrieved 2022-12-20.
  3. Oosthuizen, Megan (2022-12-20). "Artist Shows Us What A Live-Action Movie Could Look Like". fortressofsolitude.co.za. Fortress Entertainment. Retrieved 2023-02-10.
  4. Ango, Stephan (2022-07-03). "A Camera for Ideas". stephanango.com. Retrieved 2023-02-10.
  5. "synthesis". etymonline.com. Online Etymology Dictionary. Retrieved 2022-12-27.
  6. "-graphy". etymonline.com. Online Etymology Dictionary. Retrieved 2022-12-27.
  7. Underwood, Ted (2021-10-21). "Mapping the latent spaces of culture". tedunderwood.com. tedunderwood.com. Retrieved 2023-02-06.
  8. Steinbrück, Alexa (2021-08-03). "VQGAN+CLIP - How does it work?". alexasteinbruck.medium.com. medium.com. Retrieved 2023-02-06.
  9. Smith, Ethan. "A Traveler's Guide to the Latent Space". notion.com. Retrieved 2023-02-06.
  10. Roose, Kevin (2022-10-21). "A Coming-Out Party for Generative A.I., Silicon Valley's New Craze". nytimes.com. Retrieved 2023-02-06.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.