# Halloween Rorschach animations

Here are Mathematica notebooks discussing the process in detail:

Here is the link to the Imgur gallery of animations: “Attempts to recreate Rorschach’s mask”.

Here is the Halloween animation I made today:

Here is the black-&-white version:

Here is a collage of the “guiding images” for the animations above:

# Nightcore “Schweine” video making

## Introduction

This notebook/document shows how to make Nightcore modifications to a song video. We use Glukoza’s song “Schweine”. The song “Schweine” became popular via the radio station Vladivostok FM of the game “Grand Theft Auto IV”.

Remark: We use Schweine since its licencing allows copies of it to be (publicly) played via YouTube.

The Nightcore transformation of the song was fairly straightforward with Mathematica / WL. The video transformation and combination are also fairly straightforward or easy.

## Get movies

Here is a link to a version with English subtitles: https://www.youtube.com/watch?v=9Es1nPWzJ-0 .

Download at least one of the videos. (Use a Firefox or Google Chrome plugin; use VLC; or utilize the paclet described in the post “Playing with YouTube from Mathematica”, [BMI1].)

``vdSubSchweine = Import["~/Downloads/Glukoza Nostra - Schweine -subtitled-.mp4"]``

## Make Nightcore audio

The process for making a song Nightcore is described in Wikipedia, [Wk1]. Basically, we just make the tempo 20-30% faster and raise the pitch with, ≈5.5 semitones.

Remark: An alternative of the process shown in this section is to use audio transformation programs like Audacity and AmadeusPro.

Here we get the audio from the video:

``auSchweine = Audio[vdSubSchweine]``

Here we change the tempo to be 20% faster:

``````AbsoluteTiming[
auSchweineFaster = AudioTimeStretch[auSchweine, 1/1.2]
]``````

Here we raise the pitch with 5.5 semitones:

``````AbsoluteTiming[
auSchweineNightcore = AudioPitchShift[auSchweineFaster, Quantity[5.5, "Semitones"]]
]``````

## Get lyrics

Although, we have a video with English subtitles, it would be interesting to experiment with adding subtitles to the video or “discovering” the subtitles in the video frames.

Instead of just copy-&-pasting the text I took screenshot of lyrics here:
https://lyrics-on.net/en/1023698-schweine-shvajjne-lyrics.html .

Here the image above is split into two halves and they displayed in a grid:

``````imgLyrics1 = ImageTake[imgLyrics, All, {1, ImageDimensions[imgLyrics][[2]]/2}];
imgLyrics2 = ImageTake[imgLyrics, All, {ImageDimensions[imgLyrics][[2]]/2, -1}];
GraphicsGrid[{{imgLyrics1, imgLyrics2}}, Dividers -> All, ImageSize -> 700]``````

Here we recognize the lyrics within each half:

``Grid[{{TextRecognize[imgLyrics1, Language -> "Russian"], TextRecognize[imgLyrics2, Language -> "English"]}}, Dividers -> All]``

Remark: Because we found a video with subtitles, we do not use further the extracted lyrics in this notebook.

## Direct video styling

If we only wanted to change how the video looks we can directly manipulate the video frames with VideoFrameMap, [WRI6] :

``````AbsoluteTiming[
k = 0;
vdSchweine4 = VideoFrameMap[Switch[Mod[k++, 500] < 250, True, EdgeDetect[#], False, ImageEffect[#, "EdgeStylization"]] &, vdSubSchweine];
]``````
``vdSchweine4``

Remark: Since we want to make both the audio and video shorter we have to use video frames.

## Make Nightcore video

Get the frames of the video:

``````AbsoluteTiming[
lsFrames = VideoExtractFrames[vdSubSchweine, All];
]

(*{11.5501, Null}*)``````

Show the number of frames:

``````Length[lsFrames]

(*9041*)``````

Change all the frames to have the “Sepia” image effect:

``````AbsoluteTiming[
lsFramesSepia = ParallelMap[ImageEffect[#, "Sepia"] &, lsFrames];
]

(*{124.898, Null}*)``````

Generate (audio-less) video from the list of frames that have the same length as the generated audio:

``````AbsoluteTiming[
vdSubSchweineNew = VideoGenerator[lsFramesSepia, Duration[auSchweineNightcore]];
]

(*{115.34, Null}*)``````

Combine the video and audio (into a new video):

``````AbsoluteTiming[
vdSubSchweineNightcore = VideoCombine[{vdSubSchweineNew, auSchweineNightcore}];
]

(*{0.576532, Null}*)``````
``vdSubSchweineNightcore``

Remark: Here we do not export the video, since Mathematica saves it in a “standard” location of the host operating system.

## References

[BMA1] b3m2ma1, “Playing with
, (2018), Wolfram Community. (GitHub
.)

[Wk1] Wikipedia entry, “Nightcore”.

[WRI1] Wolfram Research (2010), TextRecognize, Wolfram Language
function, https://reference.wolfram.com/language/ref/TextRecognize.html
(updated 2020).

[WRI2] Wolfram Research (2016), Audio, Wolfram Language function,
https://reference.wolfram.com/language/ref/Audio.html (updated
2020).

[WRI3] Wolfram Research (2016), AudioTimeStretch, Wolfram Language
function,
https://reference.wolfram.com/language/ref/AudioTimeStretch.html
(updated 2020).

[WRI4] Wolfram Research (2016), AudioPitchShift, Wolfram Language
function,
https://reference.wolfram.com/language/ref/AudioPitchShift.html (updated
2020).

[WRI5] Wolfram Research (2020), VideoExtractFrames, Wolfram Language
function,
https://reference.wolfram.com/language/ref/VideoExtractFrames.html.

[WRI6] Wolfram Research (2020), VideoFrameMap, Wolfram Language
function, https://reference.wolfram.com/language/ref/VideoFrameMap.html
(updated 2021).

[WRI7] Wolfram Research (2008), ImageEffect, Wolfram Language
function, https://reference.wolfram.com/language/ref/ImageEffect.html
(updated 13).

[WRI8] Wolfram Research (2020), VideoGenerator, Wolfram Language
function, https://reference.wolfram.com/language/ref/VideoGenerator.html
(updated 2021).

[WRI9] Wolfram Research (2020), VideoCombine, Wolfram Language
function,
https://reference.wolfram.com/language/ref/VideoCombine.html.