Intro

Recently Rusticl has implemented shared virtual memory (SVM), see Karol Herbst’s post & related Phoronix news.

This enables SYCL on Rusticl on e.g. RadeonSI, which may be compared to SYCL on ROCm; before that, let’s take a first-time look.

Although for now Rusticl on RadeonSI has only coarse-grained SVM but AdaptiveCpp requires fine-grained SVM (author thought it isn’t too strict, see AdaptiveCpp#1113), it is possible to emulate USM upon coarse-grained SVM using intel/opencl-intercept-layer.

Build

Firstly, download source code.

git clone https://gitlab.freedesktop.org/karolherbst/mesa.git -b rusticl/svm/coarse cd mesa 

Build MESA with Rusticl enabled:

meson setup build -Dprefix="/media/build-cache/mesa-install" -Dgallium-rusticl=true -Dllvm=enabled -Dgallium-opencl=icd -Drust_std=2021 -Dgallium-drivers="radeonsi,virgl,svga,swrast,iris,crocus,i915,zink" ninja ninja install 

Note, for now MESA requires LLVM 19, and meson may select corrent Clang...

• • • >>

Ref: https://leosaa.medium.com/build-the-latest-kernel-package-for-debian-85b09ec6af88

cp -v /boot/config-$(uname -r) .config
make menuconfig
make olddefconfig

Disable signing keys and debug:

scripts/config --disable SYSTEM_TRUSTED_KEYS
scripts/config --disable SYSTEM_REVOCATION_KEYS
scripts/config --disable DEBUG_INFO
scripts/config --enable DEBUG_INFO_NONE

Note, currently rust support requires MODVERSION to be disabled, double check if Rust supports is shown in menuconfig.

Check setup of rust by:

make -j`nproc` LOCALVERSION=-rust LLVM=-19 rustavailable

where -19 is version of Clang used.

Then build:

make -j`nproc` LOCALVERSION=-rust LLVM=-19 bindeb-pkg

当导入数据到 CASA (Common Astronomy Software Applications) 时, 若台站不在 CASA 数据库中, 会得到如下报错:

RuntimeError: Exception: Telescope 21CMA is not recognized by CASA.
... thrown by casacore::MPosition casacore::MSMetaData::getObservatoryPosition(casacore::uInt) const at File: /source/casa6/casatools/casacore/ms/MSOper/MSMetaData.cc, line: 3305

从报错位置 MSMetaData.cc#L3305 可以追踪到 MeasTable.cc#L2905, MeasTable.cc#L2863, 得到关键词 “Observatories”, “measures.observatory.directory”, “geodetic”.

经过一些搜索后注意到该文件是 casacore 下载得到的, 通常位于 ~/.casa/data/geodetic/Observatories, 并且是类似 measurement set 的结构, 那么可用 Table 相关函数编辑.

注意 CASA 启动会加载该文件, 也就是加上读锁, 然后就无法获得写锁 (会死锁, 表现为程序卡死), 可以复制一份为 Observatories-bak 作为备份, 再复制一份为 Observatories-edit 编辑.

在 CASA 中使用

browsetable(tablename="/home/user/.casa/data/geodetic/Observatories-edit")

编辑加入台站参数即可. 记得退出 CASA 之后把 Observatories-edit 改回 Observatories.

需求: 用 ASCII 字符作为用户文件夹的名称, 使得命令行比较方便, 并且其他位置 (比如用户界面) 的语言维持现状.
Motivation: use ASCII characters for user folder names (thus easier when using command line) while keeping other language settings (UI text, etc.)

解决方案:
Solution:

export LC_ALL=en_US.utf8
xdg-user-dirs-gtk-update

After upgrading from 22.04 to 24.04 through do-release-upgrade, GNOME keyring no longer manages ssh key passwords when using Plasma. (Still working on GNOME).

After some digging, it is noticed that SSH_ASKPASS is set to /usr/bin/ksshaskpass and didn’t find where it is set; however ssh-askpass isn’t working properly.

As said in https://wiki.archlinux.org/title/KDE_Wallet#Using_the_KDE_Wallet_to_store_ssh_key_passphrases, will use this script in .bash_profile as a workaround:

# https://wiki.archlinux.org/title/KDE_Wallet#Using_the_KDE_Wallet_to_store_ssh
if [ "$SSH_ASKPASS" == "/usr/bin/ksshaskpass" ]; then
  export SSH_ASKPASS_REQUIRE=prefer
fi

• • • >>

本文假定安装软件到 $PREFIX, 比如 PREFIX=/opt/$NAME. 当然也可将不同软件安装到同一目录, 例如 PREFIX=$HOME/pulsar_software. 以下内容中请把 $PREFIX 替换为你想要的值.
This article assumes installing software into $PREFIX, e.g. PREFIX=/opt/$NAME. Of course you can install into same directory, e.g. PREFIX=$HOME/pulsar_software. Please replace $PREFIX with the value you want in the following content.

本文以类 Debian 环境为例, 使用其他包管理器的用户可自行搜索软件包名. 对于 RHEL, 通常 -dev -> -devel.
This article uses Debian-like environment. Users with other package managers may search for package name by yourself. For RHEL, usually -dev -> -devel.

如有任何问题或评论, 可在 https://github.com/fxzjshm/fxzjshm.github.io/issues 提出.
If you have any question or comment, consider post it at https://github.com/fxzjshm/fxzjshm.github.io/issues.

相关环境变量 | Useful environment variables

• PATH
• LD_LIBRARY_PATH
• LIBRARY_PATH
• CPATH

构建需要的软件 | Dependencies of building

• • • >>

UCX bechmark

env vars

Refer to their FAQ: https://openucx.readthedocs.io/en/master/faq.html

• debug:

  export UCX_LOG_LEVEL=info
  export UCX_PROTO_INFO=y
  

• select device:

  export UCX_NET_DEVICES=rocep65s0:1  # or sth. like mlx5_0:1
  

  get device info from ucx_info -d & lstopo

• select protocol (RoCE, TCP, etc.)
  • note: for GPU RDMA, must enable rocm_copy/cuda_copy, otherwise ucx_perftest cannot allocate GPU memory for testing

    export UCX_TLS=rc_v  # or tcp, shm, etc.
    

compile UCX

• tag v1.17.0

./autogen.sh
./contrib/configure-release --prefix=/opt/ucx --enable-mt --with-verbs --enable-devel-headers --enable-examples --enable-cma --with-cuda=/usr/local/cuda --with-rocm=/opt/rocm

host memory: RoCEv2 w1 <-> w2 <-> w3 <-> w4 <-> w5 <-> w1: 10125 MB/s

• move network controller to appropriate PCIe position
• enable RoCEv2 (priority flow control, etc.) on switch
  • e.g. on Dell switch we used, see https://www.dell.com/support/manuals/en-us/dell-emc-smartfabric-os10/smartfabric-os-user-guide-10-5-2-6/configure-roce-on-the-switch?guid=guid-ddfa4455-8f64-4014-8543-ceb6719c904d&lang=en-us

    /opt/ucx/bin/ucx_perftest -c 0
    

    /opt/ucx/bin/ucx_perftest -t ucp_put_bw 172.21.1.11 -s 1073741824 -c 3 -w 3 -n 10000
    

• • • >>