tensorflow

Installation

whq-48, docker, gpu

run:

docker run -it gcr.io/tensorflow/tensorflow bash

 

SDN

iou2, docker, ryu, ovs, mininet

http://osrg.github.io/ryu-book/en/html

run command in mininet, to access node, use

s1 ovs-ofctl -O OpenFlow13 dump-flows “s1”

where s1 indicate the node you want to process, following the command, the objective name in the command needs quote.

 

 

Related

Data

rank-number  of dimensions.

shape-list of dimensions.

 

Tutorial

http://ufldl.stanford.edu/tutorial/supervised/LinearRegression/

Advertisements
tensorflow

bugs note

Python:

pip doesn’t work:

reinstall:

#find pip
which pip
#uninstall and reinstall
/usr/local/bin/pip uninstall pip 
sudo apt-get remove python-pip
sudo apt-get install python-pip
#change permission of packages to public 
#show the location of the package
sudo pip show <package_name>
sudo chmod -R ugo+rX path_to_package
 

 

bugs note

paper process

Clustering:

1st target conf: IMC, deadline May 11,18.

2nd target conf: INFOCOMM, deadline July. need to corporate with Yordanos, Yuming.

one point: control cache which is too autonomous originally

http://jet-stream.com/blog/cache-server-vs-cdn-server/

get the syntax mapping coordinates distribution as the public source

simulate cdn with different intra-cluster similarity

ICN broker

ICN conf. deadline: May1, 8.

Get rid of search engine

Nokia lab, deadline: May 1.

paper process

vim

1.move to previous line end

To move to the beginning of the next line, hit Enter. See :help <CR>.

To make right and l wrap to the beginning of the next line, add > and l to the ‘whichwrap’ option, e.g.,

:set whichwrap+=>,l

Similarly, to make left and h wrap to the end of the previous line, add < and h to the ‘whichwrap’ option, e.g.,

:set whichwrap+=<,h

See :help 'whichwrap'.

2.

vim

OpenNetVM

Installation

https://github.com/sdnfv/openNetVM/blob/master/docs/Install.md

  1. doesn’t support osx
  2. installed on pc in Helsinki:  whq-48.cs.helsinki.fi:/home/pengzhou/openNetVM
    1. PC_NIC: Ethernet controller: Intel Corporation Ethernet Connection I217-LM (rev 04)
      $lshw            #show the hardware config
      $lshw -class network  #show the network hardware config
      $lspci             #show all pci
    2. kernel: 3.13.0
  3. echo export ONVM_NUM_HUGEPAGES=2048 >> ~/.bashrc

    #default ONVM_NUM_HUGEPAGES is 1024, here we set as 2048

  4.  export ONVM_NIC_PCI=" 00:19.0"

    #specify NIC port to be bound to DPDK, port found in step 2.

  5. run scripts/install.sh every time of rebooting, as it loads the appropraite kernel modules and can bind your NIC ports to the DPDK driver.
    https://github.com/sdnfv/openNetVM/blob/master/scripts/install.sh
    output:

    Build complete [x86_64-native-linuxapp-gcc]
    
    Installation cannot run with T defined and DESTDIR undefined
    
    Configuring 2048 hugepages with size 2048
    
    Adding huge fs to /etc/fstab
    
    Mounting hugepages
    
    Creating 2048 hugepages
    
    Configuring environment
    
    Please export $ONVM_HOME and set it to /home/pengzhou/openNetVM

    should due to

    106 scripts/setup_environment.sh
    $echo export ONVM_HOME=/home/pengzhou/openNetVM >> ~/.bashrc
    $source ~/.bashrc

    then installed successfully

  6. now here
    ./onvm/go.sh 0,1,2,3 1 -s stdout

    output:

    sudo: /home/pengzhou/openNetVM/onvm/onvm_mgr/onvm_mgr//onvm_mgr: command not found

    coming from last line of go.sh:

    sudo $SCRIPTPATH/onvm_mgr/onvm_mgr/$RTE_TARGET/onvm_mgr -l $cpu -n 4 --proc-type=primary ${virt_addr} -- -p ${ports} ${num_srvc} ${def_srvc} ${stats}

    was because of an error edit in bashrc.

  7. test completed successfully

Examples

  1. Linear NF Chain
    Determine CPU architecture and running limits:

    scripts/corehelper.py

    output:

          - openNetVM can handle 0 NFs on this system
    scripts/corehelper.py:
    182 print "** MAKE SURE HYPERTHREADING IS DISABLED **

    to disable hyper theading(sth. like virtual cpu core), {Hyper-threading (HT) refers to the hardware-backed capacity of Intel CPUs to efficiently switch between two execution threads}read file /proc/cpuinfo, in which the processors with same core ids are the real-virtual pair of cores in one physical core.
    http://nicolas.limare.net/pro/notes/2014/11/26_ht_freq_scal/
    after emailing with Neel, seems there is a bug.
    try to continue with manually command.
    using and reading onvm/go.sh:

    18 cpu=$1    #use man bash to see detail

    after running the commands, get  to the step of “start a packet generator”
    install Pktgen-DPDK.
    when running it, met with the prob about no free hugepage , should due to that opennetvm is still running, maybe should make hugepage larger in the start.

 

 

 

OpenNetVM

ipv6

The 128 bits of an IPv6 address are represented in 8 groups of 16 bits each. Each group is written as four hexadecimal digits and the groups are separated by colons (:). An example of this representation is 2001:0db8:0000:0000:0000:ff00:0042:8329.

For convenience, an IPv6 address may be abbreviated to shorter notations by application of the following rules.

  • One or more leading zeroes from any groups of hexadecimal digits are removed; this is usually done to either all or none of the leading zeroes. For example, the group 0042 is converted to 42.
  • Consecutive sections of zeroes are replaced with a double colon (::). The double colon may only be used once in an address, as multiple use would render the address indeterminate. RFC 5952 recommends that a double colon must not be used to denote an omitted single section of zeroes.[38]

An example of application of these rules:

Initial address: 2001:0db8:0000:0000:0000:ff00:0042:8329
After removing all leading zeroes in each group: 2001:db8:0:0:0:ff00:42:8329
After omitting consecutive sections of zeroes: 2001:db8::ff00:42:8329
ipv6

python

Difference between _, __ and __xx__ in Python

16 Sep 2010

When learning Python many people don’t really understand why so much underlines in the beginning of the methods, sometimes even in the end like __this__! I’ve already had to explain it so many times, it’s time to document it.

One underline in the beginning

Python doesn’t have real private methods, so one underline in the beginning of a method or attribute means you shouldn’t access this method, because it’s not part of the API. It’s very common when using properties:

class BaseForm(StrAndUnicode):
    ...
def _get_errors(self):
    &quot;Returns an ErrorDict for the data provided for the form&quot;
    if self._errors is None:
        self.full_clean()
    return self._errors

errors = property(_get_errors)</code></pre></figure>

This snippet was taken from django source code (django/forms/forms.py). This means errors is a property, and it’s part of the API, but the method this property calls, _get_errors, is “private”, so you shouldn’t access it.

Two underlines in the beginning

This one causes a lot of confusion. It should not be used to mark a method as private, the goal here is to avoid your method to be overridden by a subclass. Let’s see an example:

class A(object):
    def __method(self):
        print "I'm a method in A"
def method(self):
    self.__method()

a = A() a.method()

The output here is

$ python example.py 
I'm a method in A

Fine, as we expected. Now let’s subclass A and customize __method

class B(A):
    def __method(self):
        print "I'm a method in B"

b = B() b.method()

and now the output is…

$ python example.py
I'm a method in A

as you can see, A.method() didn’t call B.__method() as we could expect. Actually this is the correct behavior for __. So when you create a method starting with __ you’re saying that you don’t want anybody to override it, it will be accessible just from inside the own class.

How python does it? Simple, it just renames the method. Take a look:

a = A()
a._A__method()  # never use this!! please!
$ python example.py
I'm a method in A

If you try to access a.__method() it won’t work either, as I said, __method is just accessible inside the class itself.

Two underlines in the beginning and in the end

When you see a method like __this__, the rule is simple: don’t call it. Why? Because it means it’s a method python calls, not you. Take a look:

>>> name = "igor"
>>> name.len()
4
>>> len(name)
4

>>> number = 10 >>> number.add(20) 30 >>> number + 20 30

There is always an operator or native function that calls these magic methods. The idea here is to give you the ability to override operators in your own classes. Sometimes it’s just a hook python calls in specific situations. __init__(), for example, is called when the object is created so you can initialize it. __new__() is called to build the instance, and so on…

Here’s an example:

class CrazyNumber(object):
def __init__(self, n):
    self.n = n

def __add__(self, other):
    return self.n - other

def __sub__(self, other):
    return self.n + other

def __str__(self):
    return str(self.n)

num = CrazyNumber(10) print num # 10 print num + 5 # 5 print num - 20 # 30

Another example:

class Room(object):
def __init__(self):
    self.people = []

def add(self, person):
    self.people.append(person)

def __len__(self):
    return len(self.people)

room = Room() room.add("Igor") print len(room) # 1

The documentation covers all these special methods.

Conclusion

Use _one_underline to mark you methods as not part of the API. Use __two_underlines__ when you’re creating objects to look like native python objects or you wan’t to customize behavior in specific situations. And don’t use __just_to_underlines, unless you really know what you’re doing!

 a, b = b, a+b

Doing:

a, b = b, a+b

is equivalent to:

temp = a
a = b
b += temp

It lets you simultaneously do two calculations without the need of an intermediate/temporary variable.

The difference is that in your second piece of code, when you do the second line b = a+b, you have already modifed a in the previous line which is not the same as the first piece of code.

Examples

>>> a = 2
>>> b = 3
>>> a,b
2 3
>>> a,b = b,a
>>> a,b
3 2

On the other hand, if you use the second approach shown in your question:

>>> a = 2
>>> b = 3
>>> a,b
2 3
>>> a = b
>>> b = a
>>> a,b
3 3

class instantiation

The instantiation operation (“calling” a class object) creates an empty object. Many classes like to create objects with instances customized to a specific initial state. Therefore a class may define a special method named __init__(), like this:

def __init__(self):
    self.data = []

When a class defines an __init__() method, class instantiation automatically invokes __init__() for the newly-created class instance. So in this example, a new, initialized instance can be obtained by:

x = MyClass()

Of course, the __init__() method may have arguments for greater flexibility. In that case, arguments given to the class instantiation operator are passed on to __init__(). For example,

>>>

>>> class Complex:
...     def __init__(self, realpart, imagpart):
...         self.r = realpart
...         self.i = imagpart
...
>>> x = Complex(3.0, -4.5)
>>> x.r, x.i
(3.0, -4.5)

Class and Instance Variables

Generally speaking, instance variables are for data unique to each instance and class variables are for attributes and methods shared by all instances of the class:

class Dog:

    kind = 'canine'         # class variable shared by all instances

    def __init__(self, name):
        self.name = name    # instance variable unique to each instance

>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.kind                  # shared by all dogs
'canine'
>>> e.kind                  # shared by all dogs
'canine'
>>> d.name                  # unique to d
'Fido'
>>> e.name                  # unique to e
'Buddy'

As discussed in A Word About Names and Objects, shared data can have possibly surprising effects with involving mutable objects such as lists and dictionaries. For example, the tricks list in the following code should not be used as a class variable because just a single list would be shared by all Dog instances:

class Dog:

    tricks = []             # mistaken use of a class variable

    def __init__(self, name):
        self.name = name

    def add_trick(self, trick):
        self.tricks.append(trick)

>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.add_trick('roll over')
>>> e.add_trick('play dead')
>>> d.tricks                # unexpectedly shared by all dogs
['roll over', 'play dead']

Correct design of the class should use an instance variable instead:

class Dog:

    def __init__(self, name):
        self.name = name
        self.tricks = []    # creates a new empty list for each dog

    def add_trick(self, trick):
        self.tricks.append(trick)

>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.add_trick('roll over')
>>> e.add_trick('play dead')
>>> d.tricks
['roll over']
>>> e.tricks
['play dead']

* and **

The syntax is the * and **. The names *args and **kwargs are only by convention but there’s no hard requirement to use them.

You would use *args when you’re not sure how many arguments might be passed to your function, i.e. it allows you pass an arbitrary number of arguments to your function. For example:

>>> def print_everything(*args):
        for count, thing in enumerate(args):
...         print '{0}. {1}'.format(count, thing)
...
>>> print_everything('apple', 'banana', 'cabbage')
0. apple
1. banana
2. cabbage

Similarly, **kwargs allows you to handle named arguments that you have not defined in advance:

>>> def table_things(**kwargs):
...     for name, value in kwargs.items():
...         print '{0} = {1}'.format(name, value)
...
>>> table_things(apple = 'fruit', cabbage = 'vegetable')
cabbage = vegetable
apple = fruit

You can use these along with named arguments too. The explicit arguments get values first and then everything else is passed to *args and **kwargs. The named arguments come first in the list. For example:

def table_things(titlestring, **kwargs)

You can also use both in the same function definition but *args must occur before **kwargs.

You can also use the * and ** syntax when calling a function. For example:

>>> def print_three_things(a, b, c):
...     print 'a = {0}, b = {1}, c = {2}'.format(a,b,c)
...
>>> mylist = ['aardvark', 'baboon', 'cat']
>>> print_three_things(*mylist)
a = aardvark, b = baboon, c = cat

As you can see in this case it takes the list (or tuple) of items and unpacks it. By this it matches them to the arguments in the function. Of course, you could have a * both in the function definition and in the function call.

assert

Assert statements are a convenient way to insert debugging assertions into a program:

assert_stmt ::=  "assert" expression ["," expression]

The simple form, assert expression, is equivalent to

if __debug__:
    if not expression: raise AssertionError

The assert statement exists in almost every programming language. When you do…

assert condition

… you’re telling the program to test that condition, and trigger an error if the condition is false.

In Python, it’s roughly equivalent to this:

if not condition:
    raise AssertionError()

sys.version_info

A tuple containing the five components of the version number: major, minor, micro, releaselevel, and serial.

The version_info value corresponding to the Python version 2.0 is (2, 0, 0, 'final', 0). The components can also be accessed by name, so sys.version_info[0] is equivalent to sys.version_info.major and so on.

Encode

https://docs.python.org/2/howto/unicode.html
http://stackoverflow.com/questions/2241348/what-is-unicode-utf-8-utf-16

>>> from Crypto.Cipher import AES
>>> obj = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456')
>>> message = "The answer is no"
>>> ciphertext = obj.encrypt(message)
>>> ciphertext
'\xd6\x83\x8dd!VT\x92\xaa`A\x05\xe0\x9b\x8b\xf1'
>>> obj2 = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456')
>>> obj2.decrypt(ciphertext)
'The answer is no'

 

package Crypto

http://pythonhosted.org/pycrypto/

[a:b]

[a,…..,b)

array.frombytes(s)

Appends items from the string, interpreting the string as an array of machine values (as if it had been read from a file using the fromfile() method).

 

array(”)

Type code C Type Python Type Minimum size in bytes
'c' char character 1
'b' signed char int 1
'B' unsigned char int 1
'u' Py_UNICODE Unicode character 2 (see note)
'h' signed short int 2
'H' unsigned short int 2
'i' signed int int 2
'I' unsigned int long 2
'l' signed long int 4
'L' unsigned long long 4
'f' float float 4
'd' double float 8

Lambdas

lambda_expr     ::=  "lambda" [parameter_list]: expression old_lambda_expr ::= "lambda" [parameter_list]: old_expression

Lambda expressions (sometimes called lambda forms) have the same syntactic position as expressions. They are a shorthand to create anonymous functions; the expression lambda arguments: expression yields a function object. The unnamed object behaves like a function object defined with

def name(arguments):
    return expression

 reduce(function, iterable[, initializer])

Apply function of two arguments cumulatively to the items of iterable, from left to right, so as to reduce the iterable to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). The left argument, x, is the accumulated value and the right argument, y, is the update value from the iterable. If the optional initializer is present, it is placed before the items of the iterable in the calculation, and serves as a default when the iterable is empty. If initializer is not given and iterable contains only one item, the first item is returned.

Bitwise operators in Python

Operator Meaning Example
& Bitwise AND x& y = 0 (0000 0000)
| Bitwise OR x | y = 14 (0000 1110)
~ Bitwise NOT ~x = -11 (1111 0101)
^ Bitwise XOR x ^ y = 14 (0000 1110)
>> Bitwise right shift x>> 2 = 2 (0000 0010)
<< Bitwise left shift x<< 2 = 42 (0010 1000)

 

>>>[2] * 3

[2, 2, 2]

int to binary

bin()

array.insert(i, x)

Insert a new item with value x in the array before position i. Negative values are treated as being relative to the end of the array.

pip install between python3 and python2

python3 -m pip install -U --force-reinstall pip
python -m pip install -U --force-reinstall pip

will leave you with a pip pointing to python2, a pip2 pointing to python2, and a pip3 pointing to python3.

print()

in python3, print is a function. parenthesis is needed

 

 

python