Daudzslāņainu no teksta uz attēlu ģenerētu komiksu paneļu izstrādes tehniskā iespējamība

Abstract

Šī disertācija pēta mākslīgā intelekta radītu komiksu spēju saglabāt stāsta konsekvenci, loģisko secību un vizuālo saskaņu, tādējādi imitējot cilvēka radītu komisku. Disertācijā tiek iztirzāti komiksu mākslas unikālie izaicinājumi, izpētot komiksu īpatnības. Atbilstoši atklātajam tiek ģenerēti komiksi kopā ar padziļinātiem to aprakstiem izmantojot pašreizējās MI tehnoloģijas, konkrēti, difūzijas un no teksta-uz-tekstu modeļus. Literatūras apskatā atklājās, ka OpenAI DALL-E algoritms ir vispiemērotākais šī pētījuma mērķu sasniegšanai. Pētījuma metodoloģija ietver pielāgota GPT algoritma izstrādi, kas apstrādā lietotāja ievadīto komiksa ideju, lai izveidotu komiksa iezīmju sarakstu un atbilstošus paneļu aprakstus tālākai attēla ģenerācijai. Izmantojot stabilās difūzijas tehniku, kas integrēta DALL-E ģeneratorā, šie apraksti tiek pārvērsti trīs un vairāk paneļu komiksos. Darbā tika analizēti 100 MI ģenerēti attēli un salīdzināti ar 10 cilvēka radītiem komiksiem, izmantojot tīmekļa spēli, kurā dalībnieki izvēlas, kurš attēls, pēc viņu domām, ir cilvēka radīts. Visiem attēliem tika piešķirts Elo reitings, un lietotāja izvēles rezultāti tika izmantoti, lai palielinātu vai samazinātu reitingu katram attēlam. Tāpat tika pētīts, vai komikss satur tekstu, un kā tas ietekmē salīdzinājuma rezultātus. Secinājumi atklāj, ka 32% no MI ģenerētajiem komiksiem atrodas cilvēka komiksu Elo robežās. Turklāt viens no pieciem labākajiem reitingiem bija MI ģenerētam attēlam. Vidējais MI komiksu reitings bija 487,78, bet cilvēkiem - 622,20. Dalībnieki varēja pārliecinoši apgalvot, ka attēls ir cilvēka radīts tikai ar 35,8% precizitāti. Šie rezultāti norāda, ka daļa MI ģenerēto komiksu var tikt uztverta kā cilvēka radīti, parādot, ka ģeneratīvo algoritmu radīto attēlu kvalitāte un naratīva konsekvence var būt līdzvērtīga cilvēka. Tas pierāda vairākslāņu teksta-uz- attēla ģenerēšanas algoritmu potenciālu kā noderīgu rīku komiksu mākslas nozarē.

This thesis investigates the potential of multimodal AI systems to autonomously generate comic strips that exhibit a high degree of narrative consistency and visual coherence on par with human-created comic strips. Comics, as a storytelling medium, present unique challenges in automated generation due to their reliance on both textual narrative and visual elements. The aim of this research is to bridge the gap between text-to-image synthesis and sequential art storytelling, providing insights into the capabilities and limitations of current AI technologies in creative domains. The literature review explored the capability of existing advanced machine learning algorithms, including Generative Adversarial Networks (GANs) and diffusion models, to synthesize images that not only follow a given script but also maintain a consistent art style across sequential panels. Following the findings in the review OpenAI’s DALL-E text-to-image generative algorithm was concluded to be best fitting for the further thesis research and empirical measuring of the text-to-image feasibility as a form of comic creation. The research methodology comprises the development of a custom GPT algorithm that processes user input to create scripts and corresponding panel descriptions. Using stable diffusion techniques integrated in the DALL-E text-to-image generator, these descriptions are translated into coherent visual narratives in the form of three or more panel comic strips. A total of 100 AI generated images were chosen for further examination and comparison with 10 human created comics. The generated images were then compared to each other and human created comic strips to evaluate their perceived quality. This was facilitated through a web game where the users are presented with two random image options and must choose which is more likely to be human created until finding an image that they are confident is human created. Each comparison and participant decision were recorded. All images were assigned an Elo rating (500 points as base for AI images and 700 for human), and the comparison results were used to increase or decrease the rating for each image according to the result of the match-up. The calculated rating was later used to observe the perceived quality of each image and analyse the differences in rating distribution, allowing for statistically significant conclusions to be made. Additionally, some features of the comics were recorded during prompt generation and later using text detection algorithms. The effect of whether the comics contain text was also investigated as a factor for comparative success. The findings revealed that 32% of the AI generated comics lied within the boundaries of human comic Elo distribution when no starting disparity was introduced. Additionally, 1 of the best 5 ranked comics was AI generated. The average Elo rating of AI comics was 487.78, while it was 622.20 for humans. The participants could confidently state if an image is human with 35.8% accuracy. The research findings indicate that a part of AI generated comics can potentially be perceived as human created; therefore, showing that quality and narrative consistency of text- to-image generative algorithms can be on par with human ability, while not exceeding it. This proves the feasibility of multi-layered text-to-image generation of comic strips, and the potential of use as a tool in the art industry. Keywords: Text-to-image, Comic Strip, generative AI.