Halloween Rorschach animations

Last weekend I made and uploaded to YouTube a presentation discussing the making of Rorschach mask animations in both 2D and 3D:

Here are Mathematica notebooks discussing the process in detail:

Here is the link to the Imgur gallery of animations: “Attempts to recreate Rorschach’s mask”.

Here is the Halloween animation I made today:

Here is the black-&-white version:

Here is a collage of the “guiding images” for the animations above:

Nightcore “Schweine” video making

Introduction

This notebook/document shows how to make Nightcore modifications to a song video. We use Glukoza’s song “Schweine”. The song “Schweine” became popular via the radio station Vladivostok FM of the game “Grand Theft Auto IV”.

Remark: We use Schweine since its licencing allows copies of it to be (publicly) played via YouTube.

The Nightcore transformation of the song was fairly straightforward with Mathematica / WL. The video transformation and combination are also fairly straightforward or easy.

Remark: Here is the final result uploaded to YouTube (https://www.youtube.com/watch?v=8UsR9L3KPIU):

Get movies

Here is a link to the official video: https://www.youtube.com/watch?v=Ue5ZBe-GzSM .

Here is a link to a version with English subtitles: https://www.youtube.com/watch?v=9Es1nPWzJ-0 .

Download at least one of the videos. (Use a Firefox or Google Chrome plugin; use VLC; or utilize the paclet described in the post “Playing with YouTube from Mathematica”, [BMI1].)

Here we import the downloaded video:

vdSubSchweine = Import["~/Downloads/Glukoza Nostra - Schweine -subtitled-.mp4"]
1ow21rc06glgb

Make Nightcore audio

The process for making a song Nightcore is described in Wikipedia, [Wk1]. Basically, we just make the tempo 20-30% faster and raise the pitch with, ≈5.5 semitones.

Remark: An alternative of the process shown in this section is to use audio transformation programs like Audacity and AmadeusPro.

Here we get the audio from the video:

auSchweine = Audio[vdSubSchweine]
1uuz3gsogs93f

Here we change the tempo to be 20% faster:

AbsoluteTiming[
  auSchweineFaster = AudioTimeStretch[auSchweine, 1/1.2] 
 ]
1n7u00nxdhs7q

Here we raise the pitch with 5.5 semitones:

AbsoluteTiming[
  auSchweineNightcore = AudioPitchShift[auSchweineFaster, Quantity[5.5, "Semitones"]] 
 ]
005j44hxpz7h0

Get lyrics

Although, we have a video with English subtitles, it would be interesting to experiment with adding subtitles to the video or “discovering” the subtitles in the video frames.

Instead of just copy-&-pasting the text I took screenshot of lyrics here:
https://lyrics-on.net/en/1023698-schweine-shvajjne-lyrics.html .

0jnweyevtrddx

Here the image above is split into two halves and they displayed in a grid:

imgLyrics1 = ImageTake[imgLyrics, All, {1, ImageDimensions[imgLyrics][[2]]/2}];
imgLyrics2 = ImageTake[imgLyrics, All, {ImageDimensions[imgLyrics][[2]]/2, -1}];
GraphicsGrid[{{imgLyrics1, imgLyrics2}}, Dividers -> All, ImageSize -> 700]
0nok801otpf32

Here we recognize the lyrics within each half:

Grid[{{TextRecognize[imgLyrics1, Language -> "Russian"], TextRecognize[imgLyrics2, Language -> "English"]}}, Dividers -> All]
07s8wyhharkul

Remark: Because we found a video with subtitles, we do not use further the extracted lyrics in this notebook.

Direct video styling

If we only wanted to change how the video looks we can directly manipulate the video frames with VideoFrameMap, [WRI6] :

AbsoluteTiming[
  k = 0; 
  vdSchweine4 = VideoFrameMap[Switch[Mod[k++, 500] < 250, True, EdgeDetect[#], False, ImageEffect[#, "EdgeStylization"]] &, vdSubSchweine]; 
 ]
vdSchweine4
1x9obucooutc8

Remark: Since we want to make both the audio and video shorter we have to use video frames.

Make Nightcore video

Get the frames of the video:

AbsoluteTiming[
  lsFrames = VideoExtractFrames[vdSubSchweine, All]; 
 ]

(*{11.5501, Null}*)

Show the number of frames:

Length[lsFrames]

(*9041*)

Change all the frames to have the “Sepia” image effect:

AbsoluteTiming[
  lsFramesSepia = ParallelMap[ImageEffect[#, "Sepia"] &, lsFrames]; 
 ]

(*{124.898, Null}*)

Generate (audio-less) video from the list of frames that have the same length as the generated audio:

AbsoluteTiming[
  vdSubSchweineNew = VideoGenerator[lsFramesSepia, Duration[auSchweineNightcore]]; 
 ]

(*{115.34, Null}*)

Combine the video and audio (into a new video):

AbsoluteTiming[
  vdSubSchweineNightcore = VideoCombine[{vdSubSchweineNew, auSchweineNightcore}]; 
 ]

(*{0.576532, Null}*)
vdSubSchweineNightcore
0ffx6fronoawk

Remark: Here we do not export the video, since Mathematica saves it in a “standard” location of the host operating system.

References

[BMA1] b3m2ma1, “Playing with
YouTube from Mathematica”
, (2018), Wolfram Community. (GitHub
link
.)

[Wk1] Wikipedia entry, “Nightcore”.

[WRI1] Wolfram Research (2010), TextRecognize, Wolfram Language
function, https://reference.wolfram.com/language/ref/TextRecognize.html
(updated 2020).

[WRI2] Wolfram Research (2016), Audio, Wolfram Language function,
https://reference.wolfram.com/language/ref/Audio.html (updated
2020).

[WRI3] Wolfram Research (2016), AudioTimeStretch, Wolfram Language
function,
https://reference.wolfram.com/language/ref/AudioTimeStretch.html
(updated 2020).

[WRI4] Wolfram Research (2016), AudioPitchShift, Wolfram Language
function,
https://reference.wolfram.com/language/ref/AudioPitchShift.html (updated
2020).

[WRI5] Wolfram Research (2020), VideoExtractFrames, Wolfram Language
function,
https://reference.wolfram.com/language/ref/VideoExtractFrames.html.

[WRI6] Wolfram Research (2020), VideoFrameMap, Wolfram Language
function, https://reference.wolfram.com/language/ref/VideoFrameMap.html
(updated 2021).

[WRI7] Wolfram Research (2008), ImageEffect, Wolfram Language
function, https://reference.wolfram.com/language/ref/ImageEffect.html
(updated 13).

[WRI8] Wolfram Research (2020), VideoGenerator, Wolfram Language
function, https://reference.wolfram.com/language/ref/VideoGenerator.html
(updated 2021).

[WRI9] Wolfram Research (2020), VideoCombine, Wolfram Language
function,
https://reference.wolfram.com/language/ref/VideoCombine.html.