heuripedes

Callbacks, threads and thead-local storage on pybind11 2.2.4

2022-06-24T00:00:00+00:00

Disclaimer: This was also posted in the pybind discussion board

In pybind11 2.2.4, invoking a Python callback from a C++ function running in a Python-created thread will result in a situation where the Python callback will not have access to thread-local storage initialised on the same thread that invoked the C++ function:

$ pip install cppimport "pybind11==2.2.4" &>/dev/null && rm cpp_module*cpython*so && python tlstest.py 2>/dev/null
Current pybind11 version is 2.2.4
From module..........: tls.value = hello world
From thread (Python).: tls.value = hello world
From thread (C++)....: tls.value = <undefined>

The issue was probably resolved by PR #1211 (see this test code from the PR) which got merged into pybind 2.3.0:

$ pip install cppimport "pybind11==2.3.0" &>/dev/null && rm cpp_module*cpython*so && python tlstest.py 2>/dev/null 
Current pybind11 version is 2.3.0
From module..........: tls.value = hello world
From thread (Python).: tls.value = hello world
From thread (C++)....: tls.value = hello world

Commands to reproduce

For pybind 2.2.4: ` pip install cppimport “pybind11==2.2.4” &>/dev/null && rm cpp_modulecpythonso && python tlstest.py 2>/dev/null`
For pybind 2.3.0: pip install cppimport "pybind11==2.3.0" &>/dev/null && rm cpp_module*cpython*so && python tlstest.py 2>/dev/null

Code to reproduce:

Note 1: make sure you have python-dev (I’m using 3.9) and a compiler.
Note 2: copy even the comments otherwise cppimports might not be able to correctly build the native module.

`cpp_module.cpp`

// cppimport
#include <pybind11/pybind11.h>
#include <functional>

namespace py = pybind11;

std::function<void()> g_cb;

PYBIND11_MODULE(cpp_module, m) {
    m.def("set_callback", [](py::object cb) {
        g_cb = [cb]() {
            py::gil_scoped_acquire _;
            cb();
        };
    });

    m.def("call_the_callback", []() { 
        py::gil_scoped_release _;
        g_cb();
    });
}

/*
<%
setup_pybind11(cfg)
%>
*/

`tlstest.py`

import functools
import sys
import threading

import pybind11

import cppimport.import_hook
import cpp_module

tls = threading.local()
tls.value = "hello world"

def run_in_thread(f):
    @functools.wraps(f)
    def thread_wrapper(*args, **kwargs):
        def wrapper(*args, **kwargs):
            tls.value = "hello world"
            f()

        t = threading.Thread(target=wrapper, args=args, kwargs=kwargs)
        t.start()
        t.join()

    return thread_wrapper


def print_tls_value():
    print(f"tls.value = {getattr(tls, 'value', '<undefined>')}")

@run_in_thread
def called_from_python():
    print_tls_value()

@run_in_thread
def called_from_cpp():
    cpp_module.call_the_callback()

print("Current pybind11 version is", pybind11.__version__)
cpp_module.set_callback(print_tls_value)

sys.stdout.write("From module..........: ")
print_tls_value()

sys.stdout.write("From thread (Python).: ")
called_from_python()

sys.stdout.write("From thread (C++)....: ")
called_from_cpp()

Surprise order sensitivity in Python

2021-09-02T00:00:00+00:00

I’m in a research project that uses Python as the primary language for implementations and I’ve run into a very weird problem twice now. The issue is that some modules turn your code order sensitive for no apparent reason.

Here’s some snippets from a script that declares Celery tasks. TO ME, identical from a behavior standpoint. However, the first one doesn’t print anything. Why? I don’t know.

    @shared_task(bind=True)
    def start(self):
        print("Hello world")
        navegador = webdriver.Remote('http://firefox:4444/wd/hub',desired_capabilities=DesiredCapabilities.FIREFOX)
        # ...

    @shared_task(bind=True)
    def start(self):
        navegador = webdriver.Remote('http://firefox:4444/wd/hub',desired_capabilities=DesiredCapabilities.FIREFOX)
        print("Hello world")
        # ...

I ran into this issue earlier this year using OpenCV’s DNN module. In this case, running a subprocess after forwarding the network led to repeated results. I even posted a code to reproduce the problem in an issue on the opencv repo, but at the time of the writing this nobody has posted a definitive explanation or solution.

My guess is that this is related to how subprocess is implemented on Linux or perhaps how those subprocesses get started. It could also be a weird interaction between Python subprocesses and native extension threads (OpenCV makes use of OMP, TBB and plain threading).

Project idea: ‘Feet First C: The worse guide ever’

2021-03-14T00:00:00+00:00

Lately I’ve been thinking about writing a C guide of sorts. Perhaps the worse C guide ever, because I’d like to start from a really crude and unconventional point: barely compilable C code with plenty of gotos.

I can already imagine the reaction of some people when they see something like:

main() {
    int input = read_int();
    int result = 0;

    if (input == 0)
        goto input_0;
    if (input == 1)
        goto input_1;
    
    /* bad input */
    result = -1;
    goto end;

input_1:
    result = 1;
    goto end;
input_2:
    result = 2;
    goto end;

end:
    return result;
}

From that the reader would be guided into higher and higher level constructs like if/else blocks, loops and all:

/* this: */
    int i = 0;
start:
    if (i >= 10)
        goto end;
    /* ... */
next:
    i = i + 1;

/* becomes this: */

for (int i = 0; i < 10; i = i + 1) {
    /* ... */
}

The idea is to not only teach C but also teach a general idea of how the computer runs our code. My hope is that this somehow smoothens the learning curve as the reader will be presented with very few language constructs (variables, goto, if, functions). And yes, I intend to present functions before while, for etc…

Setting up ROCm 4.0 and tensorflow-rocm on Ubuntu 20.04

2021-03-01T00:00:00+00:00

ROCm 4.0 and Ubuntu Desktop 20.04 don’t play well together out of the box so while ROCm 4.1 isn’t available, we’ll have to make do with what we have. I hope you enjoy this guide.

1. The right kernel

The first step in the process is to check your kernel version as ROCm doesn’t yet support kernels newer than 5.4. Ubuntu Server installs are usually fine in this department as they ship with the GA (General Availability) stack, but Ubuntu Desktop from 20.04 onwards will ship with the Hardware Enablement Stack unless you’re using one of those HP/Dell/etc prebuilts, in which case Ubuntu will try to use the OEM flavour.

Check your kernel version with

$ uname -a

If the version reported starts with 5.4 then you can jump to step #2. Otherwise, run the following commands to downgrade to the general availability kernel but first read this note from Ubuntu wiki:

It is advised to keep Ubuntu Desktop 20.04 LTS with the kernel flavour picked during installation. It can be either HWE or OEM flavour. Changing to track GA [General Availability] kernel may result in regressions of performance, hardware support, and certified features.

sudo apt install --install-recommends linux-generic

Reboot, interrupt grub, go into Advanced options and boot the 5.4 kernel. Check that everything works as expected.

If everything is fine, get rid of the HWE/OEM kernel:

sudo apt remove --purge linux-generic-hwe-20.04 linux-oem-20.04 linux-hwe-* linux-oem-* linux-modules-5.1* linux-modules-5.8.0-* linux-modules-5.6.0-* 

2. ROCm Setup

ROCm is only available from AMD’s ROCm repository so you’ll have to add it’s PGP keys and make the repository url visible to apt.

First download and verify the key:

wget -q -O /tmp/rocm.gpg.key https://repo.radeon.com/rocm/rocm.gpg.key
echo 'e85a40d1a43453fe37d63aa6899bc96e08f2817a /tmp/rocm.gpg.key' | sha1sum -c

If the check failed, please consult the official installation guide or comment on the post (no promises). If everything went right, proceed with these commands to configure apt and install the ROCm driver and libraries:

sudo apt-key add /tmp/rocm.gpg.key
echo 'deb [arch=amd64] https://repo.radeon.com/rocm/apt/debian/ xenial main' | sudo tee /etc/apt/sources.list.d/rocm.list
sudo apt update
sudo apt install rocm-dkms rocm-libs

If the install succeeded, add the rocm executable paths to your $PATH and add yourself to the video and render groups:

echo 'export PATH=$PATH:/opt/rocm/bin:/opt/rocm/rocprofiler/bin:/opt/rocm/opencl/bin' | sudo tee -a /etc/profile.d/rocm.sh
sudo gpasswd -a $LOGNAME render
sudo gpasswd -a $LOGNAME video

Reboot your system.

After the reboot, check if ROCm is working by runnning rocm-smi and clinfo. Your GPU should be listed in them. If you can’t see your GPU, consult the official installation guide for more information.

3. Tensorflow

Tensorflow is available under two packages, tensorflow-rocm and tensorflow-rocm-enhanced. From a programming standpoint they are identical as far as I can tell so pick whatever you want:

# preferrably from a virtualenv
pip install tensorflow-rocm-enhanced

The following command will show the devices recognized by tensorflow:

python -c 'from tensorflow.python.client import device_lib; print(device_lib.list_local_devices())'

Big Blue Button: webcam, chat and presentation in one video

2020-05-01T00:00:00+00:00

It is known BBB doesn’t support downloading a recording as a single file. The main reason for this is that it ruins the experience (no jumping to specific slides) and is not flexible (can’t rearrange or resize content); the other reason is that this kind of content and usability requirements increase complexity and this is the reason the playback page exists.

While I personally really enjoy the playback page, I’m constantly bombarded with can you send me the recording? questions all the time. I’m tired of having to say no or giving lengthy explanations on why I can’t give the users a single video file they can download or store on portable storage. This motivated me to take a look on how 2.0 BBB’s published recordings work and this post is my solution to the problem. It only handles basic usage (no pan/zoom, no cursor and no deskshare), but it covers the most common use case in my experience and I hope it’ll be useful to someone else as much as it has been to me.

import xml.etree.ElementTree as ET
import cairocffi
import pangocffi
import pangocairocffi
import os.path
import shlex
import subprocess
import sys

The shapes

In the published recording folder, BBB stores when and what slide was displayed in the shapes.svg file. This file has a bunch of <image> tags that describe when and what file should be displayed (fields file, in and out). Here we’ll parse shapes.svg and generate an appropriate input file for FFmpeg’s concat demuxer which we’ll be using later to generate a video out of the presentation.

def parse_shapes(directory):
    tree = ET.parse(os.path.join(directory, "shapes.svg"))
    root = tree.getroot()
    
    shapes = []
    for x in root.findall('{http://www.w3.org/2000/svg}image'):
        shapes.append({
            'start': float(x.attrib["in"]),
            'end': float(x.attrib["out"]),
            'file': os.path.realpath(os.path.join(directory, x.attrib["{http://www.w3.org/1999/xlink}href"])),
        })
    return shapes

concat expects a file with the syntax bellow. The duration field is optional but we have to use it or else the slides will just flick for a frame or two.

file 'file1'
duration 2
file 'file2'
duration 15

def write_shape_inputs(directory, shapes):
    try:
        os.mkdir(os.path.join(directory, "temp"))
    except FileExistsError:
        pass
    
    with open(os.path.join(directory, "temp", "shapes.in"), "w") as f:
        for shape in shapes:
            filename = shape["file"].replace("'", r"\'")
            print(f"file '{filename}'", file=f)
            print(f"duration {int(shape['end'] - shape['start'])}", file=f)

The chat problem

BBB stores the chat contents in slides_new.xml as a series of <chattimeline> tags containing information about who said what and when (fields name, text and in). My guess by looking at the target field of those tags is that this file is also used for other things, such as questions, but we don’t care about them here so we’ll just make sure we only parse entries targeting chat.

def parse_chat(directory):
    tree = ET.parse(os.path.join(directory, "slides_new.xml"))
    root = tree.getroot()

    messages = []
    
    for x in root.findall("chattimeline"):
        if x.attrib["target"] == "chat":
            messages.append({
                'timestamp': int(x.attrib["in"]),
                'name': x.attrib["name"],
                'text': x.attrib["message"]
            })
    
    return messages

So far so good, but how do we display this text on the video? FFmpeg’s drawtext seemed like a good idea at first, but getting hundreds of messages to display at specific times proved challenging even using the timeline editing feature and I don’t want to get into how to get text scrolling.

I worked around the issue by rendering the messages to png files and then generating a file that the concat demuxer could use to give us a video we could use later.

def write_chat_frames(directory, messages):
    try:
        os.mkdir(os.path.join(directory, "temp"))
    except FileExistsError:
        pass
    
    surf = cairocffi.ImageSurface(cairocffi.FORMAT_ARGB32, 640, 540)
    context = cairocffi.Context(surf)
    layout = pangocairocffi.create_layout(context)
    layout.set_width(pangocffi.units_from_double(636)) # 2px before and after
    
    def format_msg(msg):
        return f"<b>{msg['name']}</b>: {msg['text']}\r\n"
    
    last_timestamp = 0
    markup = ""
    chat_frames = []
    for i in range(0, len(messages)):
        cur_timestamp = messages[i]['timestamp']
        if last_timestamp != cur_timestamp:
            context.rectangle(0, 0, 640, 540)
            context.set_source_rgb(1, 1, 1)
            context.fill()

            context.set_source_rgb(0, 0, 0)
            layout.set_markup(markup)

            height = pangocffi.units_to_double(layout.get_extents()[0].height)
            context.move_to(2, 540 - height)
            pangocairocffi.show_layout(context, layout)

            filename = os.path.realpath(os.path.join(directory, "temp", f"chat_{i}.png"))
            chat_frames.append((filename, cur_timestamp - last_timestamp))

            with open(filename, "wb") as image_file:
                surf.write_to_png(image_file)

        markup += format_msg(messages[i])

        last_timestamp = cur_timestamp
    
    return chat_frames

def write_chat_input(directory, chat_frames):
    try:
        os.mkdir(os.path.join(directory, "temp"))
    except FileExistsError:
        pass
    
    with open(os.path.join(directory, "temp", "chat.in"), "w") as f:
        for filename, duration in chat_frames:
            filename = filename.replace("'", r"\'")
            print(f"file '{filename}'", file=f)
            print(f"duration {duration}", file=f)    

Wrapping it up

Time to put the code to work for us now. All functions assume you’ll be passing a path to a folder generated by BBB’s publishing process, this means the webcam video will get encoded twice but that’s a price I’m willing to pay – that video is tiny anyways. You’ll probably find those folders inside /var/bigbluebutton/published/presentation.

We’ll start by generating the chat frames and concat demuxer files for both chat and slides:

# directory where the published presentation is
PRESENTATION_DIR = "977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811"

# for education purposes, only encode the first 10 minutes. make it [] for the real thing
EXTRA_OPTS = ["-t", "600"]

print("Generating shape.in....")
shapes = parse_shapes(PRESENTATION_DIR)
write_shape_inputs(PRESENTATION_DIR, shapes)
print(f"Processed {len(shapes)} frames")

print("Generating chat frames and chat.in...")
messages = parse_chat(PRESENTATION_DIR)
chat_frames = write_chat_frames(PRESENTATION_DIR, messages)
write_chat_input(PRESENTATION_DIR, chat_frames)
print(f"Processed {len(chat_frames)} frames")

Generating shape.in....
Processed 41 frames
Generating chat frames and chat.in...
Processed 102 frames

Now what’s left to do is to convert all the pictures to videos. The chat frames are all ready to go but we’ll need to use the scale and pad FFmpeg filters during the encode in order to fit the slides in 1280x1080.

shapes_in = os.path.join(PRESENTATION_DIR, "temp", "shapes.in")
shapes_mp4 = os.path.join(PRESENTATION_DIR, "temp", "shapes.mp4")

print("Generating shapes.mp4... (this might take a while)")
with open(os.path.join(PRESENTATION_DIR, "temp", "shapes_mp4.log"), "w") as logfile:
    cmd = [ "ffmpeg", "-y", "-hide_banner",
           "-f", "concat", "-safe", "0", "-i", shlex.quote(shapes_in),
           "-vf", "scale='min(1280,iw)':min'(1080,ih)':force_original_aspect_ratio=decrease,pad=1280:1080:(ow-iw)/2:(oh-ih)/2",
           "-preset", "veryfast", "-tune", "stillimage", "-pix_fmt", "yuv420p"]
    cmd = cmd + EXTRA_OPTS + [shlex.quote(shapes_mp4)]
          
    
    print(f"    {' '.join([shlex.quote(c) for c in cmd])}")
    proc = subprocess.run(cmd, stdout=logfile, stderr=logfile, universal_newlines=True)
print("Done")

Generating shapes.mp4... (this might take a while)
    ffmpeg -y -hide_banner -f concat -safe 0 -i 977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811/temp/shapes.in -vf 'scale='"'"'min(1280,iw)'"'"':min'"'"'(1080,ih)'"'"':force_original_aspect_ratio=decrease,pad=1280:1080:(ow-iw)/2:(oh-ih)/2' -preset veryfast -tune stillimage -pix_fmt yuv420p -t 600 977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811/temp/shapes.mp4
Done

chat_in = os.path.join(PRESENTATION_DIR, "temp", "chat.in")
chat_mp4 = os.path.join(PRESENTATION_DIR, "temp", "chat.mp4")

print("Generating chat.mp4... (this might take a while)")
with open(os.path.join(PRESENTATION_DIR, "temp", "chat_mp4.log"), "w") as logfile:
    cmd = ["ffmpeg", "-y", "-hide_banner", "-f", "concat", "-safe", "0", "-i", shlex.quote(chat_in),
           "-preset", "veryfast", "-tune", "stillimage", "-pix_fmt", "yuv420p"]
    cmd = cmd + EXTRA_OPTS + [shlex.quote(chat_mp4)]
    
    print(f"    {' '.join([shlex.quote(c) for c in cmd])}")
    proc = subprocess.run(cmd, stdout=logfile, stderr=logfile, universal_newlines=True)
print("Done")

Generating chat.mp4... (this might take a while)
    ffmpeg -y -hide_banner -f concat -safe 0 -i 977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811/temp/chat.in -preset veryfast -tune stillimage -pix_fmt yuv420p -t 600 977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811/temp/chat.mp4
Done

BBB’s webcams.webm file has a resolution of 640x480 and the sole reason to pick 640x540 for the chat and 1280x1080 for the shapes is to be able to lay all three out in the following configuration while outputting a 1920x1080 video:

######################
#             # cams #
#             #      #
#  slides     ########
#             # chat #
#             #      #
#             #      #
######################

The command to do so will follow soon but first I want to explain the value I passed in -filter_complex. I split it up in separate filter chains so it’s a little easier to read:

[0:v]pad=1920:1080:x=0:y=-1[padded];: pick the video stream from input #0 (shapes.mp4), pad it to 1920x1080 and keep it centerd vertically. Name the output as [padded].
[padded][1:v]overlay=x=1280:y=480[overlaid];: Take [padded] and lay the video stream from input #1 (chat.mp4) over it at the coordinates (1280, 480). Name the output of this operation as [overlaid].
[overlaid][2:v]overlay=x=1280[out]: Take [overlaid] and lay the video stream from input #2 (webcams.webm) over it at the coordinates (1280, 0). Name the output as [out] (this will end up in the output file with the -map argument).

webcams_webm = os.path.join(PRESENTATION_DIR, "video", "webcams.webm")
conference_mp4 = os.path.basename(PRESENTATION_DIR) + "_output.mp4"

print(f"Generating {conference_mp4}... (this might take even longer than the previous two)")
with open(os.path.join(PRESENTATION_DIR, "temp", "output_mp4.log"), "wb") as logfile:
    cmd = ["ffmpeg", "-y", "-i", shlex.quote(shapes_mp4), "-i", shlex.quote(chat_mp4),
           "-i", shlex.quote(webcams_webm),
           "-filter_complex", "[0:v]pad=1920:1080:x=0:y=-1[padded];[padded][1:v]overlay=x=1280:y=480[overlaid];[overlaid][2:v]overlay=x=1280[out]",
           "-map", "[out]", "-c:v", "libx264", "-preset", "slow", "-profile:v", "high", "-level", "4.0", "-movflags", "+faststart",
           "-crf", "18", "-bf", "2", "-coder", "1",
           "-map", "2:a", "-c:a", "aac", "-ac", "2", "-cpu-used", "0"]
    cmd = cmd + EXTRA_OPTS + [shlex.quote(conference_mp4)]
    
    print(f"    {' '.join([shlex.quote(c) for c in cmd])}")
    proc = subprocess.run(cmd, stdout=logfile, stderr=logfile, universal_newlines=True)

print("All done!")

Generating 977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811_output.mp4... (this might take even longer than the previous two)
    ffmpeg -y -i 977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811/temp/shapes.mp4 -i 977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811/temp/chat.mp4 -i 977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811/video/webcams.webm -filter_complex '[0:v]pad=1920:1080:x=0:y=-1[padded];[padded][1:v]overlay=x=1280:y=480[overlaid];[overlaid][2:v]overlay=x=1280[out]' -map '[out]' -c:v libx264 -preset slow -profile:v high -level 4.0 -movflags +faststart -crf 18 -bf 2 -coder 1 -map 2:a -c:a aac -ac 2 -cpu-used 0 -t 600 977d4a145cd70f8b013d723cedbab0ba4b8bba74-1585911377811_output.mp4
All done!

The output file will be named after the published directory name (<directory>_output.mp4) and stored in the current working directory. All other files we created are in the temp folder inside the published recording directory. It’s safe to delete this folder and if you ever need to debug FFmpeg invocation issues check the .log files inside.

If you want to turn this into an script, just copy and paste all the code snippets into a .py file and change the PRESENTATION_DIR to something more appropriate (perhaps sys.argv[1]?).

I hope this post helped. Have a good worker’s day!

Old pamphlet: A New Money for a New Age

2019-11-24T00:00:00+00:00

Transcription of an old pamphlet I found on the internet in the night of 24/11/2019

A New Money for a New Age

Free on Application to E. W. ABRAHAM

112 Faithful Street
Victoria, B.C.

A New Money for a New Age

In view of the world-wide search for a new Monetary System which will bring real Security, the following thoughts on the Economics of the Bible are submitted, not with the intention of being dogmatic, nor as hard as fast rules, but as suggestions for the ground-work of further study along these lines. If abundance and plenty are worth having, they are worth studying and working for.

The subject will be discussed under nine headings, as follows:

The Kind of Government we need to administer a Righteous Economic System.
The Form of Money we would require.
The Law against Usury.
The Sabbatic Law of Release.
Provision for the Poor, Sick, Young, Old, Widows, etc.
A Minimum Wage.
Pensions for ALL.
Taxes, or tithe.
Government Control.

1. The Kind of Government We need…

First and foremost, we MUST HAVE a Government we can RELY UPON. A government made up, as Moses instructed, of “able men, such as fear God, men of Truth, hating covetousness…” Such a Government’s duties would be not to MAKE its own laws, but to ADMINISTER the economic laws of the Bible, Ex. 18.21; Deut. 1, 15-17; and 16, 18.

2. The Form of Money Required

Money would be provided ONLY by the Government, and SOLELY as a MEANS OF EXCHANGE. The Banks would not be allowed to create money in ANY FORM WHATEVER. Money would be SERVANT of the people, instead of its MASTER; issued as required, by the Government, and backed by the needs and resources of the country, for the USE and BENEFIT of our own people. Surely that is not too much to ask in this land of plenty – ENOUGH TOKENS OF EXCHANGE to enable us to PROPERLY EMPLOY ALL our people, and to PROPERLY care for our children, old people, and the sick – that is only common-sense after all. We certainly have not been using common-sense when we have allowed our children to be undernourished, our sick and aged neglected, and thousands of our people unemployed, just because we have NOT ENOUGH MONEY; although no one will suggest for one moment that we have not enough resources to fill ALL our needs. All money issued would have to be in some UNHOARDABLE FORM, i.e. dated or limited in some way as to avoid any possibility of its being hoarded, or held off the market 1 Tim. 6:10.

3. The Law Against Usury

USURY MUST BE ENTIRELY ABOLISHED. Under Bible economics, usury of ANY KIND is forbidden within the nation. With a Monetary System controlled by a RIGHTEOUS Government, with money for use PURELY as a MEANS OF EXCHANGE, there would be NO ROOM for usury or interest. The Bible makes NO DISTINCTION between usury and interest. Deut 23.19, reads: “Thous shalt not lend upon usury to thy brother; usury of money, usury of victuals, usury of ANYTHING THAT IS LENT UPON USURY…” and this injunction was confirmed again and again. Read Lev. 25.36; Ezekiel 18, 5 to 9; and many other passages.

We might, perhaps, notice, in passing, that “no interest” does not mean “no dividends” upon legitimate investments. The difference between interest on a loan, and dividends on an investment, is that the latter is paid ONLY when there is a profit to pay with it, whereas interest is payable even if the business run at a loss.

4. The Sabbatic Law of Release

The Bible is authority for an economic Sabbath, called the Law of Release, (Lev. 25, 3/4: Deut. 15, 1) whereby ALL debts shall be canceled every seventh year. God states that He will assume any debt canceled under this clause. (The probability is that as God constantly uses human means for the fulfillment of His Purposes, there would be a Central Fund out of tax surplus, whereby any canceled debts would be paid by the State.) It seems obvious that this debt clause applies chiefly to the poor and needy, and is not intended as an escape for dishonest people.

5. Provision For The Sick, Poor, Aged, Etc.

All hospitals, old peoples’ homes, asylums, etc., would be FREE and Government operated. Today our whole national economy, our health, our wealth, our moral standards, our very freedom, are all dependent upon “money.” There would be no suffering humanity as we see it today, if we lived up to Deut. 7, which reads: “Therefore shalt thou keep the Commandments, and the Statutes, and the Ordinances which I command thee this day… And it shall come to pass… that the Lord thy God will keep unto thee the Covenant and the Kindness which He hath shewn unto thy fathers… AND THE LORD WILL TAKE AWAY FROM THEE ALL SICKNESS…”

6. A Minimum Wage

Our Lord’s Parable of the Labourers in the vineyard (Matt. 20) would appear to give full authority for a Minimum Wage. “Whatsoever is right, “ the Master of the vineyard said to the men hired at the eleventh hour, “that shall ye receive;” and He gave them THE SAME WAGE as those who had been working all day, showing plainly that He recognized that their “right” or “need” was equal with those who had been hired earlier, even though their opportunity for earning ta wage had not been so great. In this age of statistics, it should be a simple matter to determine the minimum wage NEEDED in any country – it should be SUFFICIENT to provide the worker with ALL the necessities of life. Thereafter, of course the wage would be determined by the natural law of supply and demand, together with the efficiency of the worker. The Bible age for workers appears to be from 20 years old to 55, and in some cases from 25 years old to 50.

St. Paul clearly provided the answer to those who say that this system would be abused by the lazy, when he says in 2 Thess. 3, 10; “He that will not work, neither shall he eat.” There is also an age-old axiom, given by God Himself to Adam (Gen. 3, 19) “In the sweat of thy face shall thou eat bread, till thou return unto the ground.”

7. Pensions for All

Under a righteous SERVANT-MONEY SYSTEM, ALL citizens, upon reaching retiring age (generally 55), would automatically retire and receive their first pension cheque, and could then look forward to a happy and provided-for old age, without having to work right up to the edge of the grave, as so many do nowadays. Here again the Statistical Branch would be able to work out the amount of the ideal pension – based upon a supply of NECESSITIES for the rest of one’s life. (Probably it would be found to be a somewhat similar amount to the Minimum Wage). Proof of age and citizenship would be the ONLY REQUIREMENT for eligibility – the Bureau of Vital Statistics would keep in touch with every man, woman and child in the country, and would pay the pension for ALL, from the highest to the lowest in the land, as soon as it became due. There would be no need for property or means test. Deut. 10.10; 24.17; Ex. 22.22; Jer. 7, 5/6.

8. Taxes or Tithe

Only ONE TAX for everyone. 10% of ALL Income or increase. (In exceptional cases 20% may be expected of the wealthy). In the case of there being no increase, as with a farmer’s poor crop for instance, there would be no tax at all that year – or if the income were to cease for any reason, so would the tax. One-tenth of all taxes would go to the upkeep of the Church, the remaining nine-tenths to the State for all purposes, such as education, hospitals, pensions, etc. Mal. 3.8/11 etc.

9. The Government Control NEEDED

All essential services, such as light, heat, water, transportation, should be run by the Government for the benefit of the people; though under the Bible Economic System, where no huge combines, monopolies and cartels could flourish, and where private enterprise could be better relied upon to serve the people at a reasonable rate, this may not be necessary. It does not require very much imagination to visualize the future, with atomic power harnessed for all sorts of BENEFICIAL USES; central heating and lighting for whole cities; unlimited power for farms and factories, or for transportation, etc. Today there is LITERALLY NOTHING standing in the way of these wonders, except finance, and our own sin and greed, and refusal to obey God’s Laws.

Please do not toss this article on the side, with a shrug of the shoulders and the idea that we have been discussing Utopia. GOD DOES NOT MOCK HIS PEOPLE. A perfectly just and equitable Monetary System is both feasible and possible, and is, without a doubt to be found in the Bible. It is up to us to study it out, but above all to trust God, and to ask Him to raise up righteous men amongst us WHO WILL PUT IT INTO FORCE.

E. W. ABRAHAM
Victoria, B.C., 1946

Old pamphlet: How To Increase Wages Without Raising Prices

2019-11-24T00:00:00+00:00

Transcription of an old pamphlet I found on the internet in the night of 24/11/2019

How To Increase Wages Without Raising Prices

THE BIBLE SOLUTION TO THE LABOR-MANAGEMENT DISPUTE
by ALEXANDER SCHIFFNER

When men are under subjection to another’s will and command without adequate compensation for services rendered it is slavery no matter by what name such service may be called, for slavery is the involuntary servitude of one human being to another.

No question has aroused so much interest as the present labor-management dispute. The failure to offer a solution satisfactory to both industry and labor has led to strife and bloodshed. But it is impossible to make an equitable adjustment of wages which will be satisfactory to both employer and employee until certain fundamental changes are made in our economy.

Both employers and employees are under pressure today – the former endeavoring to remain in business; the latter to meet ever increasing living costs.

The money needed by the employer to meet his payrolls and to purchase his stock is furnished by the money lenders to whom he must pay tribute. Too, he must pay dividends, earned by the employees, to the stockholders who through no labor on their part receive the fruits of another’s labor.

Our national payroll is approximately 150 billion dollars annually. Another 75 billion dollars (approximate) is paid to non-producers in the form of dividends on their investments or to financiers in discounts or interest. Labor therefore produces 225 billion dollars worth of goods annually but receives only a part of what it produces.

Why should a third party who contributes nothing, either in brains or brawn, receive the fruits of another’s labor? In collecting tribute for the financing of production, the producer is robbed of his profits and the laborer of his wages.

When industry shakes itself loose from the present enslaving usury system, labor and management alike will receive the full fruitage of their labor. Money was never meant to be a producer. It is not a producer. Yet we permit it to produce for thousands at the expense of the workers. The Bible says “that if any would not work, neither should he eat.” 2 Thes. 3:10 Labor needs the employer who in turn needs labor and its services, yet neither actually needs the financial non-producers.

Vested financial interests have been able to remain aloof from most labor disputes, making both management and labor assume that their difficulties are the result of the greed of the laborer or the selfishness of the employer, when in fact both are dupes and slaves of a financial system which reaps both ends and the middle.

STOP PAYING TO THIS THIRD PARTY THAT FOR WHICH IT HAS NEVER WORKED OR LABORED AND WAGES CAN BE INCREASED WITHOUT RAISING PRICES.

This parasite third party should be as much concerned about bringing about a change in our economy as in labor and management. Especially since we are witnessing the fulfillment of James 5:1-5. Read it. Publication and distribution of this and similar tracts is made possible by your tithes and free-will offerings. “This gospel of the Kingdom shall be preached.” – Matt. 24:13.

ADDRESS
BETHEL TRACT SOCIETY
Box 7 – Spokane I, Wash.

O que fazer quando passar de ano antes do ano acabar

2019-11-13T00:00:00+00:00

Se passou antes do fim, ainda há o que aprender.
Faça valer a oportunidade,
já que muitos não a tem assim,
de estudar com tranquilidade
aquilo que ainda há de te render.

Se passou antes do fim, não se gabe.
Ajude o colega que se debate
numa situação ruim.
Pois o futuro ninguém sabe
e o colega, bem tradado,
mesmo terrivelmente cansado,
pode ser que salve
este teu umbiguim.

CSCamp CTF 2013 Forensics2 Writeup

2013-11-16T00:00:00+00:00

enygmata/fmul: Alright guys, first-off I want to say that this text does not really show everything I did to beat this challenge. It took me about four hours to figure everything out and I don’t remember most of the things i tried.

Forensics 2

We got a memory dump of a challenger accused of cheating, can you confirm that?

The challenge description provides a memory dump of a VMware machine (.vmem). Googling around for memory analysers and related things I found the volatility project, which has a Python script to analyse memory dumps and other stuff.

Before doing any real work, vol.py (the volatility script) needs a profile that describes the kind of data we are going to handle. Because of the filename, I already knew it was a Windows XP machine but since it was the first time using vol.py I just followed the instructions:

$ vol.py -f Windows\ XP\ Professional-f67cc587.vmem imageinfo
Volatile Systems Volatility Framework 2.2
Determining profile based on KDBG search...

          Suggested Profile(s) : WinXPSP2x86, WinXPSP3x86 (Instantiated with WinXPSP2x86)
                     AS Layer1 : JKIA32PagedMemoryPae (Kernel AS)
                     AS Layer2 : FileAddressSpace (<edited>/Windows XP Professional-f67cc587.vmem)
                      PAE type : PAE
                           DTB : 0x34c000L
                          KDBG : 0x8054d2e0L
        Number of Processors : 2
     Image Type (Service Pack) : 3
                KPCR for CPU 0 : 0xffdff000L
                KPCR for CPU 1 : 0xf8942000L
             KUSER_SHARED_DATA : 0xffdf0000L
           Image date and time : 2013-10-17 23:19:02 UTC+0000
     Image local date and time : 2013-10-18 00:19:02 +0100

Now we can take a look at the dump. Let’s see which processes the guy was running:

$ vol.py -f Windows\ XP\ Professional-f67cc587.vmem pslist
Volatile Systems Volatility Framework 2.2
Offset(V)  Name                    PID   PPID   Thds     Hnds   Sess  Wow64 Start                Exit
---------- -------------------- ------ ------ ------ -------- ------ ------ -------------------- --------------------
0x823c6660 System                    4      0     61      281 ------      0
0x81fcb020 smss.exe                548      4      3       19 ------      0 2013-06-30 09:22:16
0x81f3e020 csrss.exe               672    548     12      370      0      0 2013-06-30 09:22:23
0x820f3020 winlogon.exe            696    548     19      582      0      0 2013-06-30 09:22:23
0x8205c020 services.exe            740    696     15      294      0      0 2013-06-30 09:22:23
0x81fdc020 lsass.exe               752    696     20      359      0      0 2013-06-30 09:22:23
0x820e97f0 vmacthlp.exe            940    740      1       25      0      0 2013-06-30 09:22:25
0x8208d6e8 svchost.exe             956    740     16      207      0      0 2013-06-30 09:22:26
0x81e5e020 svchost.exe            1004    740      8      296      0      0 2013-06-30 09:22:26
0x82072da0 svchost.exe            1148    740     68     1388      0      0 2013-06-30 09:22:26
0x820137a8 svchost.exe            1340    740      5       78      0      0 2013-06-30 09:22:26
0x820e93c8 svchost.exe            1448    740     12      189      0      0 2013-06-30 09:22:28
0x81ee2620 spoolsv.exe            1780    740     10      140      0      0 2013-06-30 09:22:29
0x82164da0 explorer.exe           1828   1736     10      461      0      0 2013-06-30 09:22:29
0x81f3d3b8 rundll32.exe            128   1828      4       74      0      0 2013-06-30 09:22:32
0x81f3cc08 vmtoolsd.exe            140   1828      5      199      0      0 2013-06-30 09:22:32
0x81e6cda0 svchost.exe             360    740      5      111      0      0 2013-06-30 09:22:44
0x820f1c10 svchost.exe             396    740      5      105      0      0 2013-06-30 09:22:44
0x81fcf4b8 vmtoolsd.exe            592    740      7      278      0      0 2013-06-30 09:22:45
0x82230350 imapi.exe               516    740      4      118      0      0 2013-06-30 09:22:59
0x821efc10 alg.exe                1428    740      6      110      0      0 2013-06-30 09:22:59
0x821e7da0 wscntfy.exe            1572   1148      1       37      0      0 2013-06-30 09:23:00
0x822c5980 wuauclt.exe            2140   1148      3      110      0      0 2013-06-30 09:24:03
0x8215a648 rundll32.exe           3548   1828      0 --------      0      0 2013-10-17 21:58:10  2013-10-17 21:58:33
0x820a5c10 ctfmon.exe             2176    732      1       88      0      0 2013-10-17 22:42:46
0x820ac5a0 cmd.exe                2504   1828      1       33      0      0 2013-10-17 22:44:26
0x82101318 xchat.exe              3348   1828      3       92      0      0 2013-10-17 22:45:34
0x822bf4d8 decryptpastebin        3292   2504      1       90      0      0 2013-10-17 23:18:43

The last process seems suspicious. I dumped it using the procmemdump vol.py command and looking at it’s strings I noticed Python and py2exe references. The next step would be analysing this executable but py2exe executables are just garbage mixed with internal Python commands so I had to find a decompiler. I found py2exe-extract:

$ py2exe-extract decryptpastebin
[*] extracting C:\Python27\lib\site-packages\py2exe\boot_common.py
[*] extracting decryptpastebin.py
[*] done!

Unfortunately that output lies, py2exe-extract did not create a .py file. It just extracted the compiled module (.pyc). Now it was time for some real decompiling action, so I used uncompyle2, but it was made for Python 2.7 and decryptpastebin.pyc had 2.6 bytecode. Luckily, uncompyle2 worked fine after I shut it’s mouth by commenting the if (lines 73 and 74 of uncompyle2/__init__.py) that checked for the bytecode version.

$ uncompyle2  decryptpastebin.pyc

$ python2 decryptpastebin.py
Welcome to pastebin decryptor v0.1beta
======================================
ID:KEY=

Looking inside decryptpastebin.py, I saw that ID was a pastebin.com paste id:

def decryptbin(idkey):
    id = idkey.split(':')[0]
    key = idkey.split(':')[1]
    import httplib
    c = httplib.HTTPConnection('pastebin.com')
    c.request('GET', '/raw.php?i=' + id)
    response = c.getresponse()
    if response.status == 200:
        data = response.read()
        print AESdecrypt(key, data, True)

I looked at decryptpastebin’s strings again but there was no compromising pastebin.com string. There was nothing interesting on the other processes’ strings, so I decided to check the vmem image’s string. A couple HTTP requests to pastebin.com came into view:

GET /raw.php?i=V5BvcN64 HTTP/1.1
Host: pastebin.com
...
GET /raw.php?i=dpAtnvCB HTTP/1.1
Host: pastebin.com
...

So, now I had two pastebin ids, V5BvcN64 and dpAtnvCB, but I needed the keys to proceed. Again, I didn’t find a thing from the executable files and had to check the .vmem file:

$ strings Windows\ XP\ Professional-f67cc587.vmem | grep V5BvcN64 | sort -u
GET /raw.php?i=V5BvcN64 HTTP/1.1
:sp4nky!~sp4nky@41.227.151.248 PRIVMSG [DDS]ahmed :this one V5BvcN64:LtqK_tRU7YlF
this one V5BvcN64:LtqK_tRU7YlF
        this one V5BvcN64:LtqK_tRU7YlF
this one V5BvcN64:LtqK_tRU7YlF
V5BvcN64:LtqK_tRU7YlF

The string V5BvcN64:LtqK_tRU7YlF matches decryptpastebin.py’s KEY:ID format so let’s try that:

$ python2 decryptpastebin.py
Welcome to pastebin decryptor v0.1beta
======================================
ID:KEY= V5BvcN64:LtqK_tRU7YlF
+====================================>
The flag is: d99cbe01e4b0c0da0e4711e7c0114e79
+====================================>

And there’s the flag.

Cuidado com o cabeçalho Host do HTTP

2013-10-27T00:00:00+00:00

A especificação da versão 1.1 do protocolo HTTP descreve um cabeçalho chamado Host, este cabeçalho contém o nome de host do site que o usuário deseja acessar. O cabeçalho serve para que o servidor web possa identificar qual recurso o usuário está requisitando quando o servidor atende múltiplos nomes de host (sites).

Apesar de exigir a presença do cabeçalho, a RFC do protocolo, no item 5.2, autoriza que o mesmo seja ignorado caso o cliente HTTP faça a requisição utilizando uma URI absoluta. Abaixo você pode ver um exemplo de requisição com URI relativa e outro com URI absoluta, respectivamente:

GET /index.php HTTP/1.1
Host: exemplo.com.br


GET http://exemplo.com.br/index.php HTTP/1.1
Host: exemplo.com.br

Ignorar o valor Host não representa um problema de segurança, exceto quando o programador assume que ele sempre representa o servidor atual. Infelizmente isso acontece bastante, muitos colocam códigos como o abaixo para diferenciar entre modo de produção e modo de desenvolvimento em seus programas:

if (preg_match('/^(localhost|127.0.0.1)(\:\d+)?/', $_SERVER['HTTP_HOST'])) {
	imprime_algo_comprometedor();
}

Para explorar a falha de segurança, é necessário que você crie uma requisição HTTP customizada que envia o host correto pela URI mas enviar localhost no cabeçalho Host. Podemos fazer isso usando telnet:

[higor@whiterice tmp]$ telnet exemplo.com.br 80
Trying 127.0.0.1...
Connected to exemplo.com.br
Escape character is '^]'.
GET http://exemplo.com.br/index.php HTTP/1.1
Host: localhost

HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 16:11:42 GMT
Server: Apache
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html

MODO DE DESENVOLVIMENTO. A SENHA DE admin É "123tr0c4r!"

Connection closed by foreign host.

E assim nós descobrimos a senha de admin.

Se você quer evitar este tipo de problema, você deve utilizar o nome do servidor ($_SERVER['SERVER_NAME']) em vez do valor de Host ($_SERVER['HTTP_HOST']). Porém ainda existe algo fazer se você utiliza o Apache para servir suas páginas, será necessário dizer ao Apache que você quer que a variavel de ambiente SERVER_NAME seja igual ao nome que você deu ao seu servidor (ServerName), pra isso você deve ativar a opção UseCanonicalName:

<VirtualHost *>
	ServerName exemplo.com.br
	UseCanonicalName on
</VirtualHost>

Se você utiliza PHP e/ou Apache, você pode ler mais sobre a solução e sobre o problema neste resposta do StackOverflow.