How to access git repository through http proxy?

  1. Install socat. E.g., on SUSE:
    zypper in socat
    
  2. Download this file:
    #!/bin/sh
    # Use socat to proxy git through an HTTP CONNECT firewall.
    # Useful if you are trying to clone git:// from inside a company.
    # Requires that the proxy allows CONNECT to port 9418.
    #
    # Save this file as gitproxy somewhere in your path (e.g., ~/bin) and then run
    #   chmod +x gitproxy
    #   git config --global core.gitproxy gitproxy
    #
    # More details at http://tinyurl.com/8xvpny
    
    # Configuration. Common proxy ports are 3128, 8123, 8000.
    _proxy=proxy.yourcompany.com
    _proxyport=3128
    
    exec socat STDIO PROXY:$_proxy:$1:$2,proxyport=$_proxyport
    
  3. Rename its name as gitproxy, and modify _proxy and _proxyport.
  4. Copy it into your PATH , e.g., /usr/bin.
  5. Switch to the directory which contains gitproxy, and execute the following commands:
    # chmod +x gitproxy
    # git config --global core.gitproxy gitproxy
    
  6. Now, git clone git://... will work!

 

A brief introduction of zypper

After installing SUSE (my version is SLES 12), the CD/DVD will be added as a default repository:

linux-uibj:~ # zypper repos -d
# | Alias       | Name        | Enabled | Refresh | Priority | Type  | URI                                                         | Service
--+-------------+-------------+---------+---------+----------+-------+-------------------------------------------------------------+--------
1 | SLES12-12-0 | SLES12-12-0 | Yes     | No      |   99     | yast2 | cd:///?devices=/dev/disk/by-id/ata-VBOX_CD-ROM_VB2-01700376 |       

So after you insert the installation ISO file into CDROM, use zypper in command can install the software:

linux-uibj:~ # zypper se systemtap
Loading repository data...
Reading installed packages...

S | Name              | Summary                              | Type
--+-------------------+--------------------------------------+-----------
  | systemtap         | Instrumentation System               | package
  | systemtap         | Instrumentation System               | srcpackage
  | systemtap-docs    | Documents and examples for systemtap | package
  | systemtap-docs    | Documents and examples for systemtap | srcpackage
  | systemtap-runtime | Runtime environment for systemtap    | package
  | systemtap-server  | Systemtap server                     | package
linux-uibj:~ # zypper in systemtap
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 3 NEW packages are going to be installed:
  libebl1 systemtap systemtap-runtime

3 new packages to install.
Overall download size: 1.4 MiB. Already cached: 0 B  After the operation, additional 5.7 MiB will be used.
Continue? [y/n/? shows all options] (y):

After removing the repository, even the ISO file is still in CDROM, the zypper in command doesn’t work:

linux-uibj:~ # zypper rr 1
Removing repository 'SLES12-12-0' ...................................................................................................[done]
Repository 'SLES12-12-0' has been removed.
linux-uibj:~ # zypper in systemtap
Warning: No repositories defined. Operating only with the installed resolvables. Nothing can be installed.

You can also add repository’s URL:

linux-uibj:~ # zypper ar http://xxx.net/mrepo/SLE-12-Server-x86_64/disc1/ SLES12-1
Adding repository 'SLES12-1' ........................................................................................................[done]
Repository 'SLES12-1' successfully added
Enabled: Yes
Autorefresh: No
GPG check: Yes
URI: http://xxx.net/mrepo/SLE-12-Server-x86_64/disc1/

linux-uibj:~ # zypper in systemtap
Building repository 'SLES12-1' cache ................................................................................................[done]
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 3 NEW packages are going to be installed:
  libebl1 systemtap systemtap-runtime

3 new packages to install.
Overall download size: 1.4 MiB. Already cached: 0 B  After the operation, additional 5.7 MiB will be used.
Continue? [y/n/? shows all options] (y):

Many *-devel packages are in SDK ISO file, so you should also add the SDK ISO into repository.

 

Use SystemTap to track forking process

The SystemTap website provides a forktracker.stp script to track the forking process flow, and the original script is like this (P.S.: now, the script has been modified):

probe kprocess.create
{
  printf("%-25s: %s (%d) created %d\n",
         ctime(gettimeofday_s()), execname(), pid(), new_pid)
}

probe kprocess.exec
{
  printf("%-25s: %s (%d) is exec'ing %s\n",
         ctime(gettimeofday_s()), execname(), pid(), filename)
}

After executing it, the output confused me:

......
Thu Oct 22 05:09:42 2015 : virt-manager (8713) created 8713
Thu Oct 22 05:09:42 2015 : virt-manager (8713) created 8713
Thu Oct 22 05:09:42 2015 : virt-manager (8713) created 8713
Thu Oct 22 05:09:43 2015 : virt-manager (8713) created 8713
......

Why the father and children processes had the same process ID: 8713. At first, I thought it was because the speciality of fork: call once, return twice. So I wrote a simple program to test whether it was due to fork:

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>

int main(void)
{
    pid_t pid;

    pid = fork();
    if (pid < 0) {
        exit(1);
    } else if (pid > 0) {
        printf("Parent exits!\n");
        exit(0);
    }

    printf("hello world\n");
    return 0;
}   

This time, the script outputed the following:

......
Thu Oct 22 05:27:10 2015 : bash (3855) created 8955
Thu Oct 22 05:27:10 2015 : bash (8955) is exec'ing "./test"
Thu Oct 22 05:27:10 2015 : test (8955) created 8956
......

The father and child had different process IDs, so it is not fork system call’s fault.

After resorting to SystemTap mailing list, Josh Stone gave me the answer, and it is related to Linux thread implementation: In Linux, the thread is actually also a process, so for a multi-thread program in Linux, you can think it as a thread-group. The whole thread-group has a thread-group-id(In SystemTap, pid() and new_pid‘ value), and every thread has a unique ID (In SystemTap, tid() and new_tid‘ value). Josh Stone also modified the script like this:

probe kprocess.create {
  printf("%-25s: %s (%d:%d) created %d:%d\n",
         ctime(gettimeofday_s()), execname(), pid(), tid(), new_pid, new_tid)
}

probe kprocess.exec {
  printf("%-25s: %s (%d) is exec'ing %s\n",
         ctime(gettimeofday_s()), execname(), pid(), filename)
}

To verify it, I wrote a multi-thread program:

#include <stdio.h>
#include <pthread.h>
void *thread_func(void *p_arg)
{
        while (1)
        {
                printf("%s\n", (char*)p_arg);
                sleep(10);
        }
}
int main(void)
{
        pthread_t t1, t2;

        pthread_create(&t1, NULL, thread_func, "Thread 1");
        pthread_create(&t2, NULL, thread_func, "Thread 2");

        sleep(1000);
        return;
}

The script output was like this:

......
Sat Oct 24 10:56:35 2015 : bash (889) is exec'ing "./test"
Sat Oct 24 10:56:35 2015 : test (889:889) created 889:890
Sat Oct 24 10:56:35 2015 : test (889:889) created 889:891
......

From the output, we can see: the main thread had the same tid() and pid() value: 889. All the threads had the same pid: 889, but every thread had unique tid values: 889, 890, 891.

Reference:
How to understand the pid() and new_pid are same value in executing forktracker.stp?.

 

Install SystemTap on Suse

The Suse is SLES(Suse Linux Enterprise Server) version.

(1) Install C/C++ Compiler and Tools:

Capture4-667x500

(2) Install SystemTap tools:

# zypper in systemtap*
......

(3) Install kernel debug info packages:

/mnt/suse/x86_64 # ls | grep kernel
kernel-default-base-debuginfo-3.12.49-3.1.x86_64.rpm
kernel-default-debuginfo-3.12.49-3.1.x86_64.rpm
kernel-default-debugsource-3.12.49-3.1.x86_64.rpm
kernel-xen-base-debuginfo-3.12.49-3.1.x86_64.rpm
kernel-xen-debuginfo-3.12.49-3.1.x86_64.rpm
kernel-xen-debugsource-3.12.49-3.1.x86_64.rpm
kernelshark-debuginfo-2.0.4-3.95.x86_64.rpm
nfs-kernel-server-debuginfo-1.3.0-13.1.x86_64.rpm
/mnt/suse/x86_64 # rpm -ivh kernel*
Preparing...                          ################################# [100%]
......

You can also use zypper in kernel-*-debug*.

(4) Test:

/mnt/suse/x86_64 # stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
Pass 1: parsed user script and 102 library script(s) using 78240virt/28440res/2708shr/26436data kb, in 160usr/20sys/184real ms.
Pass 2: analyzed script: 1 probe(s), 1 function(s), 3 embed(s), 0 global(s) using 175768virt/126996res/3688shr/123964data kb, in 1650usr/250sys/1902real ms.
Pass 3: using cached /root/.systemtap/cache/38/stap_38af4dc0b3509fcb42d451417e95bbab_1375.c
Pass 4: using cached /root/.systemtap/cache/38/stap_38af4dc0b3509fcb42d451417e95bbab_1375.ko
Pass 5: starting run.
read performed
Pass 5: run completed in 20usr/290sys/638real ms.

All is OK!