31か月前公開・31か月前更新・0 pv・13 min read

[Colabと比較も] Magicodeの実行環境を調べてみた

pythonMagicodeshell

https://cdn.magicode.io/media/notebox/1d5973ea-85b4-4711-843a-c16b3cbf6faa.jpeg

背景

Magicodeはipynb形式で記事を執筆しコードの実行も行うことができ、非常に便利なサービスです。

記事を書こうと思ったときに、とても重い処理を書いてもいいのだろうかと疑問に思い、コードの実行環境にどのような制約があり、どこまでできるのかを調べてみました。

簡単なshellを叩いてみる

まず手始めに、以下のshellを実行してみます。

pwd

現在のディレクトリの場所を教えてくれるshellです。

実行すると、/home/jovyanとわかりました。

ユーザー名にjovyanが割り当てられているようです。

!pwd

/home/jovyan

ls

現在のディレクトリ内のファイルを教えてくれます。

デフォルトで、requirements.txt test.ipynbの２つのファイルがあります。

test.ipynbが今執筆しているノートブックです。

requirements.txtはノートブックの起動時に自動的にpip installしてくれるライブラリをまとめて記載するためのファイルです。

!ls

requirements.txt test.ipynb

cat を使用してrequirements.txtの中身を見てみます。

pandasやnumpy、scikit-learnなど分析用や計算用のライブラリのほか、seabornなど可視化のライブラリも入っています。

!cat requirements.txt

# 適当なライブラリ詰め合わせ beautifulsoup4==4.6.3 matplotlib==3.2.2 multiprocess==0.70.12.2 multitasking==0.0.9 networkx==2.6.3 numpy==1.19.5 opencv-contrib-python==4.1.2.30 opencv-python==4.1.2.30 pandas==1.1.5 pystan==2.19.1.1 scikit-learn==0.22.2.post1 seaborn==0.11.2 spacy==2.2.4 tqdm==4.62.3 requests>=2.13.0,<3.0.0

/bin/配下に実行可能なshellコマンドが格納されているので、lsコマンドを使用して確認してみます。

ファイルを操作するような標準的なコマンドは使用することができることが分かります。

!ls /bin/

bash chmod findmnt lsblk rmdir uncompress bunzip2 chown grep mkdir run-parts vdir bzcat cp gunzip mknod sed wdctl bzcmp dash gzexe mktemp sh which bzdiff date gzip more sh.distrib ypdomainname bzegrep dd hostname mount sleep zcat bzexe df kill mountpoint stty zcmp bzfgrep dir less mv su zdiff bzgrep dmesg lessecho nisdomainname sync zegrep bzip2 dnsdomainname lessfile pidof tar zfgrep bzip2recover domainname lesskey ps tempfile zforce bzless echo lesspipe pwd touch zgrep bzmore egrep ln rbash true zless cat false login readlink umount zmore chgrp fgrep ls rm uname znew

実行速度を調べてみる

実行速度の確認のために、以下のgithubソースを使用します。
実行させる回数が多いと完了まで時間がかかるので、ループ回数を100000から1000に変更しています。

https://github.com/morikatron/snippet/blob/master/benchmark/simple_python_bench.py

ソースの中で、5つ指標を出力し多角的に実行速度を確認することができます。また、platformとpsutilライブラリを使用して実行環境も取得してくれます。

５つの処理

loop_test
function_call_test
calc_julia_set
ProcessPoolExecutor
ThreadPoolExecutor

まず関数化された計算処理部分を実行します。

"""
ジュリア集合の計算アルゴリズムは オライリー「ハイパフォーマンス Python」に記載されているものを使っています。
書籍: https://www.oreilly.co.jp/books/9784873117409/
Github: https://github.com/mynameisfiber/high_performance_python/blob/master/01_profiling/cpu_profiling/julia1_nopil.py
"""
import datetime
import os
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor

def loop_test(loop_num: int):
    """
    指定した回数 何もしないでループ処理を回します。
    :param loop_num: 空ループを回す回数
    :return: ループ回数
    """
    for i in range(loop_num):
        pass
    return loop_num


def func():
    return True


def function_call_test(loop_num: int):
    """
    指定した回数 何もしない関数 func を実行します。
    :param loop_num: 繰り返し回数
    :return: ループ回数
    """
    for i in range(loop_num):
        func()
    return loop_num


def print_test(loop_num: int):
    """
    指定した回数 print します。
    :param loop_num: 繰り返し回数
    :return: ループ回数
    """
    for i in range(loop_num):
        print("Hello Python World.")
    return loop_num


x1, x2, y1, y2 = -1.8, 1.8, -1.8, 1.8
c_real, c_imag = -0.62772, -0.42193

def calculate_z_serial_purepython(maxiter, zs, cs):
    """ジュリア漸化式を用いてoutput リストを計算する"""
    output = [0] * len(zs)
    for i in range(len(zs)):
        n = 0
        z = zs[i]
        c = cs[i]
        while abs(z) < 2 and n < maxiter:
            z = z * z + c
            n += 1
        output[i] = n
    return output


def calc_pure_python(desired_width, max_iterations):
    """複素数の座標リストzs と、複素数のパラメータリストcs を
    作り、ジュリア集合を作って表示する"""
    x_step = (float(x2 - x1) / float(desired_width))
    y_step = (float(y1 - y2) / float(desired_width))
    x = []
    y = []
    ycoord = y2
    while ycoord > y1:
        y.append(ycoord)
        ycoord += y_step
    xcoord = x1
    while xcoord < x2:
        x.append(xcoord)
        xcoord += x_step

    zs = []
    cs = []
    for ycoord in y:
        for xcoord in x:
            zs.append(complex(xcoord, ycoord))
            cs.append(complex(c_real, c_imag))

    output = calculate_z_serial_purepython(max_iterations, zs, cs)


def write_sys_info(file):
    """
    実行環境の情報をファイルに書き込みます。
    :param file: ファイルハンドル
    :return: None
    """
    import platform
    import psutil

    file.write("Python:\n")
    file.write(" - Python: {} ({}) \n".format(platform.python_version(), platform.python_implementation()))
    file.write(" - Build: {} \n".format(platform.python_build()))
    file.write(" - Compiler: {} \n".format(platform.python_compiler()))
    file.write("PC Info:\n")
    file.write(" - OS: {} \n".format(platform.platform()))
    file.write(" - Processor: {} \n".format(platform.processor()))
    file.write("   - Core: {}/{} \n".format(psutil.cpu_count(logical=False), psutil.cpu_count(logical=True)))
    file.write("   - Freq(MHz): {} \n".format(psutil.cpu_freq(percpu=False)))
    file.write(" - Memory: {}GB ({:,} Byte) \n".format( round( psutil.virtual_memory().total/(1024*1024*1024) ), psutil.virtual_memory().total))
    file.write("\n")

上で定義した関数を使って計算し、result.txtに結果を出力します。

"""
TurboBoost の効果なのか最初だけ高速になってしまうので
測定しやすさのために最初に無駄な計算を挟みます。
"""
calc_pure_python(desired_width=1000, max_iterations=100)

file = open("result.txt", mode="w", encoding="utf-8")
write_sys_info(file)

times = []
for i in range(10):
    start = datetime.datetime.now()
    a = loop_test(1000)
    elapsed = datetime.datetime.now() - start
    times.append(elapsed.seconds*1000 + elapsed.microseconds/1000)
file.write("loop_test: {} ms\n".format(sum(times)/len(times)))

times = []
for i in range(10):
    start = datetime.datetime.now()
    for i in range(1000):
        func()
    elapsed = datetime.datetime.now() - start
    times.append(elapsed.seconds*1000 + elapsed.microseconds/1000)
file.write("function_call_test: {} ms\n".format(sum(times)/len(times)))


times = []
for i in range(10):
    start = datetime.datetime.now()
    a = print_test(1000)
    elapsed = datetime.datetime.now() - start
    times.append(elapsed.seconds * 1000 + elapsed.microseconds / 1000)
file.write("print_test: {} ms\n".format(sum(times) / len(times)))

times = []
for i in range(10):
    start = datetime.datetime.now()
    calc_pure_python(desired_width=1000, max_iterations=100)
    elapsed = datetime.datetime.now() - start
    times.append(elapsed.seconds * 1000 + elapsed.microseconds / 1000)
file.write("calc_julia_set: {} ms\n".format(sum(times) / len(times)))

times = []
for i in range(10):
    start = datetime.datetime.now()
    with ThreadPoolExecutor(max_workers=os.cpu_count()) as executor:
        for i in range(1000):
            executor.submit(func)
    elapsed = datetime.datetime.now() - start
    times.append(elapsed.seconds * 1000 + elapsed.microseconds / 1000)
file.write("Thread: {} ms\n".format(sum(times) / len(times)))

times = []
for i in range(10):
    start = datetime.datetime.now()
    with ProcessPoolExecutor(max_workers=os.cpu_count()) as executor:
        for i in range(1000):
            executor.submit(func)
    elapsed = datetime.datetime.now() - start
    times.append(elapsed.seconds * 1000 + elapsed.microseconds / 1000)
file.write("Process: {} ms\n".format(sum(times) / len(times)))

file.close()

Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. Hello Python World. too many strings

出力結果を確認します。この結果は、後でGoogle Colabと比較します。

!cat result.txt

Python: - Python: 3.7.12 (CPython) - Build: ('default', 'Oct 26 2021 06:08:53') - Compiler: GCC 9.4.0 PC Info: - OS: Linux-5.4.0-107-generic-x86_64-with-debian-buster-sid - Processor: x86_64 - Core: 4/4 - Freq(MHz): scpufreq(current=2599.999, min=0.0, max=0.0) - Memory: 16GB (16,786,288,640 Byte) loop_test: 0.016700000000000003 ms function_call_test: 0.073 ms print_test: 66.21579999999999 ms calc_julia_set: 2744.2662 ms Thread: 12.8452 ms Process: 136.38240000000002 ms

実行環境を調べてみる

実行環境もresult.txtに出力してくれています。私が実施したタイミングで内容を確認すると、Linuxはdebianで4core、メモリは16GBのリソースが存在することが分かります。

実行リソースはユーザー個別で仮想的に分割されて使用できるのかもしれません。

Google Colabと比べてみる

Colab上でもshellを実行して比較してみます。また、実行速度を比較します。

shellの実行結果(Colab)

Colab上で実行した結果は以下のようになります。

!pwd

/content

!ls

sample_data

!ls /bin/

bash	       hostname       su
bunzip2        journalctl     sync
bzcat	       kill	      systemctl
bzcmp	       kmod	      systemd
bzdiff	       less	      systemd-ask-password
bzegrep        lessecho       systemd-escape
bzexe	       lessfile       systemd-hwdb
bzfgrep        lesskey	      systemd-inhibit
bzgrep	       lesspipe       systemd-machine-id-setup
bzip2	       ln	      systemd-notify
bzip2recover   login	      systemd-sysusers
bzless	       loginctl       systemd-tmpfiles
bzmore	       ls	      systemd-tty-ask-password-agent
cat	       lsblk	      tar
chgrp	       lsmod	      tempfile
chmod	       mkdir	      touch
chown	       mknod	      true
cp	       mktemp	      udevadm
dash	       more	      ulockmgr_server
date	       mount	      umount
dd	       mountpoint     uname
df	       mv	      uncompress
dir	       networkctl     vdir
dmesg	       nisdomainname  wdctl
dnsdomainname  pidof	      which
domainname     ps	      ypdomainname
echo	       pwd	      zcat
egrep	       rbash	      zcmp
false	       readlink       zdiff
fgrep	       rm	      zegrep
findmnt        rmdir	      zfgrep
fuser	       run-parts      zforce
fusermount     sed	      zgrep
grep	       sh	      zless
gunzip	       sh.distrib     zmore
gzexe	       sleep	      znew
gzip	       stty

実行速度を調べてみる(Colab)

Colab上の実行結果は以下のようになりました。 Magicodeと並べて比較してみます。

Colabの結果

Python:
 - Python: 3.7.13 (CPython) 
 - Build: ('default', 'Apr 24 2022 01:04:09') 
 - Compiler: GCC 7.5.0 
PC Info:
 - OS: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic 
 - Processor: x86_64 
   - Core: 1/2 
   - Freq(MHz): None 
 - Memory: 13GB (13,617,745,920 Byte) 

loop_test: 0.026299999999999997 ms
function_call_test: 0.11679999999999999 ms
print_test: 74.2027 ms
calc_julia_set: 4593.1935 ms
Thread: 20.3976 ms
Process: 277.7211 ms

Magicodeの結果（再掲）

Python:
 - Python: 3.7.12 (CPython) 
 - Build: ('default', 'Oct 26 2021 06:08:53') 
 - Compiler: GCC 9.4.0 
PC Info:
 - OS: Linux-5.4.0-107-generic-x86_64-with-debian-buster-sid 
 - Processor: x86_64 
   - Core: 4/4 
   - Freq(MHz): scpufreq(current=2599.999, min=0.0, max=0.0) 
 - Memory: 16GB (16,786,288,640 Byte) 

loop_test: 0.016700000000000003 ms
function_call_test: 0.073 ms
print_test: 66.21579999999999 ms
calc_julia_set: 2744.2662 ms
Thread: 12.8452 ms
Process: 136.38240000000002 ms

結果の比較

以下のリストに私が実行した時点のMagicode実行結果を載せています。Magicodeはリソースが共有されている場合、時間やタイミングで結果がかなりぶれる可能性もあります。参考としてデータを見ることをお勧めします。

ほとんどの指標でMagicodeの方が良い結果となりました。Colabに割り当てられるCPUコア数がMagicodeと比較して少ないことも影響していそうです。

CPUだけの処理をする場合、Colabよりも早い可能性が高いですがその差は顕著でもないため、好みのレベルになるかも知れません。


指標	Magicode	Colab
loop_test:	0.0167 ms	0.026299ms
function_call_test:	0.073 ms	0.116799ms
print_test:		66.2158 ms
calc_julia_set:	2744.2662 ms	4593.1935 ms
Thread:	12.8452 ms	20.3976 ms
Process:	136.3824 ms	277.7211 ms

追加検証

Magicode上の実行環境は固定ではなくかなり幅を持って変更されているような挙動が見られました。(一時的に、750GBのメモリが割り当てられたような結果が出たことがありました。)

挙動を知るために、複数回の実行速度結果をまとめて残すことにします。

1回目実行時

Python:
 - Python: 3.7.12 (CPython) 
 - Build: ('default', 'Oct 26 2021 06:08:53') 
 - Compiler: GCC 9.4.0 
PC Info:
 - OS: Linux-5.4.0-107-generic-x86_64-with-debian-buster-sid 
 - Processor: x86_64 
   - Core: 4/4 
   - Freq(MHz): scpufreq(current=2599.999, min=0.0, max=0.0) 
 - Memory: 16GB (16,786,288,640 Byte) 

loop_test: 0.0165 ms
function_call_test: 0.07399999999999998 ms
print_test: 69.3745 ms
calc_julia_set: 2821.5529 ms
Thread: 13.8477 ms
Process: 148.4952 ms