Open edX Devstack

Set up a WSL development environment

Install WSL

Open PowerShell (or Windows Command Prompt) and enter:

1

wsl --install

For Ubuntu or Debian, update and upgrade packages, use the command in bash:

1

sudo apt update && sudo apt upgrade

Proxy and Firewall

1
2
3
4
5

export HTTP_PROXY=http://172.22.32.1:10809 &&
export HTTPS_PROXY=http://172.22.32.1:10809 &&
export http_proxy=http://172.22.32.1:10809 &&
export https_proxy=http://172.22.32.1:10809 &&
export ALL_PROXY=http://172.22.32.1:10809

1
2
3
4

unset http_proxy &&
unset HTTP_PROXY &&
unset https_proxy &&
unset HTTPS_PROXY

Windows IP Address**

1

cat /etc/resolv.conf | grep nameserver | awk '{ print $2 }'

WSL IP Address

1

hostname -I | awk '{print $1}'

WSL access Windows

Add “allow” rule to Windows firewall for WSL2 network

Open PowerShell as Admin and enter the following command:

1

New-NetFirewallRule -DisplayName "WSL" -Direction Inbound  -InterfaceAlias "vEthernet (WSL)"  -Action Allow

Windows Access WSL

Windows access application in WSL by <wsl-ip> and <port>

Disable Microsoft Defender Firewall from PowerShell

1

Set-NetFirewallProfile -Profile Domain,Public,Private -Enabled False

Re-enable firewall from PowerShell

1

Set-NetFirewallProfile -Profile Domain,Public,Private -Enabled True

Test

1

ping <ip_address>

1

telnet <ip_address> <port_number>

Proxy

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

#!/bin/sh
hostip=$(cat /etc/resolv.conf | grep nameserver | awk '{ print $2 }')
wslip=$(hostname -I | awk '{print $1}')
port=<PORT>

PROXY_HTTP="http://${hostip}:${port}"

set_proxy(){
    export http_proxy="${PROXY_HTTP}"
    export HTTP_PROXY="${PROXY_HTTP}"

    export https_proxy="${PROXY_HTTP}"
    export HTTPS_proxy="${PROXY_HTTP}"
}

unset_proxy(){
    unset http_proxy
    unset HTTP_PROXY
    unset https_proxy
    unset HTTPS_PROXY
}

test_setting(){
    echo "Host ip:" ${hostip}
    echo "WSL ip:" ${wslip}
    echo "Current proxy:" $https_proxy
}

if [ "$1" = "set" ]
then
    set_proxy

elif [ "$1" = "unset" ]
then
    unset_proxy

elif [ "$1" = "test" ]
then
    test_setting
else
    echo "Unsupported arguments."
fi

1
2
3
4
5

git config --global http.proxy "${PROXY_HTTP}"
git config --global https.proxy "${PROXY_HTTP}"

git config --global --unset http.proxy
git config --global --unset https.proxy

v2rayN

“允许来自局域网的连接”

Install make

1

sudo apt install make

SSH key and GitHub

1

cd ~/.ssh && ssh-keygen

1

cat id_rsa.pub

Add your key to your account via the GitHub website.

1
2

git config --global user.name "GITHUB-USERNAME"
git config --global user.email "name@example.com"

Install Docker Desktop WSL 2 backend on Windows

• Download [Docker Desktop for Windows](https://desktop.docker.com/win/main/amd64/Docker Desktop Installer.exe?_gl=110t7s1l_gaNTM4MDE1OTE2LjE2ODYyODc3OTY._ga_XJWPQMJYHQ*MTY4NjI4Nzc5Ni4xLjEuMTY4NjI4ODgzNC41OC4wLjA.).

• Follow the usual installation instructions to install Docker Desktop. If you are running a supported system, Docker Desktop prompts you to enable WSL 2 during installation. Read the information displayed on the screen and enable WSL 2 to continue.

• Start Docker Desktop from the Windows Start menu.

• From the Docker menu, select Settings and then General.

• Select the Use WSL 2 based engine check box.

  If you have installed Docker Desktop on a system that supports WSL 2, this option is enabled by default.

• Select Apply & Restart.

Enabling Docker support in WSL 2 distros:

• When Docker Desktop starts, go to Settings > Resources > WSL Integration.
• Enable integration with additional distros.

change Disk image location:

• When Docker Desktop starts, go to Settings > Resources > Advanced.
• Set Disk image location from %localappdata%\Docker\wsl to D:\docker\wsl

Setup Docker Registry Mirroring

Add registry-mirrors property by going to settings > Docker Engine inside Docker Desktop application.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

{
  "builder": {
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
  "experimental": false,
  "features": {
    "buildkit": true
  },
  "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn", "http://hub-mirror.c.163.com", "https://registry.docker-cn.com"]
}

Visual Studio Code

Open VS Code and install the Remote - WSL extension. This extension allows you to work with code inside the Linux distro and your IDE client still on Windows.

Now, you can start working in VS Code remotely. To do this, open your terminal and type:

1

wsl

1

code .

Install Devstack

Service List

Setup and run

open a WSL terminal (Ubuntu for example) and enter:

1

export DEVSTACK_WORKSPACE=~/workspace

1

git clone https://github.com/edx/devstack ~/workspace/devstack

1

cd ~/workspace/devstack

1
2
3

git checkout open-release/olive.master
export OPENEDX_RELEASE=olive.master
make dev.checkout

1
2
3

git checkout open-release/nutmeg.master
export OPENEDX_RELEASE=nutmeg.master
make dev.checkout

1
2

make dev.clone.https
# or, `make dev.clone` if you have SSH keys set up.

1

unset OPENEDX_RELEASE

pull any changes made to the various images

1

make dev.pull.large-and-slow

configure the various services

1

make dev.provision

1
2
3

make lms-shell
rm /root/.pip/pip.conf
pip install setuptools_scm

1
2
3

make studio-shell
rm /root/.pip/pip.conf
pip install setuptools_scm

1

make dev.provision.services

Start services

1

make dev.up.large-and-slow

Commands

Start the desired services

1

make dev.up.<service>

To stop a service

1

make dev.stop.<service>

Stop service and remove the container

1

make dev.down.<service>

Access to one of the services

1

make dev.shell.<service>

View the logs of a specific service container

1

make dev.logs.ecommerce

Access

1

http://localhost:18000/

1

http://localhost:18010/

Build system

invoke

paver

Tasks

• task

  You designate a function as being a task by using the @task decorator.

  1 2 3 4 5

  from paver.easy import task @task def mytask(args): pass

• consume_args

  The @consume_args decorator tells Paver that all command line arguments following this task’s name should be passed to the task.

  1 2 3 4 5 6

  from paver.easy import task, consume_args @task @consume_args def help(args, help_function): pass

• consume_nargs

  specifying a number of consumed arguments

  1 2 3 4 5 6

  from paver.easy import task, consume_nargs @task @consume_nargs(3) def mytask(args): pass

• cmdopts

  1 2 3 4 5 6 7 8

  from paver.easy import task, cmdopts @task @cmdopts([ ('username=', 'u', 'Username to use when logging in to the servers') ]) def deploy(options): pass

  @cmdopts takes a list of tuples, each with long option name, short option name and help text. If there’s an “=” after the long option name, that means that the option takes a parameter. Otherwise, the option is assumed to be boolean. The command line options set in this manner are all added to the options under a namespace matching the name of the task. In the case above, options.deploy.username would be set if the user ran paver deploy -u my-user-name.

• needs

  specify that a task depends on another task running first with the @needs decorator. A given task will only run once as a dependency for other tasks.

  1 2 3 4 5 6 7 8 9

  from paver.easy import task, needs, cmdopts @task @needs(['deploy_to_linux']) @cmdopts([ ('username=', 'u', 'Username to use when logging in to the servers') ], share_with=['deploy_to_linux']) def deploy(options): pass

• call_task

  call an appropriate task in the middle of another task using call_task

  1 2 3

  from paver.easy import call_task call_task('say_hello', options={}, args = [])

• sh

  sh(command, capture=False, ignore_error=False, cwd=None, env=None)

  1 2 3

  from paver.easy import sh sh("rm -rf test_root/log/auto_screenshots/*")

  If capture is True, the output of the command will be captured and returned as a string. If the command has a non-zero return code raise a BuildFailure. You can pass ignore_error=True to allow non-zero return codes to be allowed to pass silently, silently into the night. If you pass cwd=’some/path’ paver will chdir to ‘some/path’ before exectuting the command. env is a dictionary of environment variables.

1
2
3

pavelib
	utils
		envs.py

1
2

class Env:
	environment_variable = value

1
2

pavelib
	prereqs.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

# prereqs.py
from paver.easy import call_task, cmdopts, consume_args, needs, sh, task
from .utils.envs import Env

def no_prereq_install():
    """
    Determine if NO_PREREQ_INSTALL should be truthy or falsy.
    """
    return str2bool(os.environ.get('NO_PREREQ_INSTALL', 'False'))
    
def log_installed_python_prereqs():
    """  Logs output of pip freeze for debugging. """
    sh("pip freeze > {}".format(Env.GEN_LOG_DIR + "/pip_freeze.log"))
    
def python_prereqs_installation():
    """
    Installs Python prerequisites
    """
    # 	edx-platform installs some Python projects from within the edx-platform repo itself.
    sh(f"pip install -e .")
    for req_file in PYTHON_REQ_FILES:
        pip_install_req_file(req_file)

def pip_install_req_file(req_file):
    """Pip install the requirements file."""
    pip_cmd = 'pip install -q --disable-pip-version-check --exists-action w'
    sh(f"{pip_cmd} -r {req_file}")
    
@task
@timed
def install_python_prereqs():
	...

@task
@timed
def install_prereqs():
    """
    Installs Node and Python prerequisites
    """
    if no_prereq_install():
        print(NO_PREREQ_MESSAGE)
        return

    if not str2bool(os.environ.get('SKIP_NPM_INSTALL', 'False')):
        install_node_prereqs()
    install_python_prereqs()
    log_installed_python_prereqs()

Command Line

rake

webpack

Reference

Set up a WSL development environment

Installing and Starting the Open edX Platform

Open edX Devstack