Pointers in C (demystified)

I have recently realized that I have not been posting here for almost precisely two years, which is an unforgivable pause for a blogger. Hope my readers would forgive me this. Thinking of what I should write about next, I realized that I have entirely forgotten about an essential thing. The thing, I would probably say. Which is, it is not only what I want to write about, but also what my readers may be interested in or even need assistance with.

Having published my first book last October, I plead guilty to checking the stats on Amazon, perhaps a bit too often. The good in it is that it also gives me the opportunity to look for books that would be interesting for myself. This was when I came across the “Understanding and Using C Pointers: Core Techniques for Memory Management” by Richard M Reese (the book may be found here for those interested). The first thought was “why would someone write an entire book on a thing like pointers?” But then, thinking to myself that books are not written for merely having been written, I took a glance at the free sample and took some time to read the comments.

Of course, the book is not dedicated to pointers only. It is a somewhat detailed explanation what memory means in C and, although I have not had a chance to read the whole book, I highly recommend it to those of my readers who are interested in C programming and want to do it the right way.

All this has reminded me of the fact that the topic of pointers in C is too often a problematic one for many students. I remember it forcing a few honors pupils in my class in the college scratch their heads trying to understand what it is. At the time we were childish and cruel and used to say that if something is not understood - “the problem is in the shim between the chair and the screen.” Although correct in many cases, this statement is not applicable when it comes to pointers (well, at least up to a certain level).

When it comes to pointers, the problem is mainly the way they are presented. Students are often given the information, which is utterly redundant at the time on one hand, and lack a basic understanding of the underlying architecture. And by the word “architecture” I mean both hardware and software. Clearly, the problem is the way they, students, are taught. This thread on Stack Exchange serves a perfect illustration of the fact that students are shown things in reverse order. Why not start building houses with the roof then? How is a student supposed to understand pointers if he/she does not know what the structure of the program is and how it works on lower levels. Instead, they are told how dangerous and unneeded Assembly is.

However, the idea behind this post is not to argue curricula – there are many other people smart enough to spoil it properly without the help of others. Let us concentrate on the technical aspects in an attempt to make the subject clearer.

Memory

First of all, we need to make it clear to ourselves what the computer memory is. Just to put it simply – computer memory is an array of cells each capable of storing a certain amount of information. In our case, talking about Random-Access Memory, each addressable cell is capable of storing one byte of data. It is this array of cells that holds all programs being executed at the moment and data thereof. While memory management is quite an exciting and broad topic by itself, we will not dive into it as it is merely irrelevant. Two last things worth mentioning about memory in the context of pointers are that each cell has an address, and, as we decided not to talk about memory management, we will refer to memory as the memory of a running process.

Variables

When we declare a variable in our code, regardless of whether it is Assembly or C code, we just tell the compiler to refer to the specific memory area by its alias – the name of the variable and that only a particular type of data could be stored at that area. For example:

int myInteger;

This tells the compiler to name a specific area in memory (it is the compiler that decides what area it is and we do not care about that at this point) as myInteger and that we could only store a value of type int there. From now on we may efficiently use the name of the variable (which is memory area alias) in our code, and the compiler will know which address to operate on.

The line above would be equivalent to this statement in Assembly (Flat Assembler syntax as usual) in case we are declaring a global variable (yes, in case of Assembly it does matter):

myInteger  dd ?

In case of a local variable, however, it would be a bit different:

push ebp
mov ebp, esp
sub esp, 4
virtual at ebp – 4
                myInteger dd ?
end virtual

But, in both cases, we are doing the same – telling the compiler to label specific memory area as myInteger and use it for storage of an integer value.

Sometimes, however, we need to know what the address of a variable is. For example, if we expect some function to modify the content of a local variable in addition to returning some value, in which case the variable is out of the scope of that function.

Pointers

Imagine the following situation – we have an unsigned integer counter and a function that increments it. The function returns 0 all the time unless an overflow occurs, meaning the counter has been reset to 0. The prototype would be something like this:

int increment_counter(unsigned int* counter);

As we see, the “unsigned int* counter” declares a function parameter as a pointer to an integer. Which, in turn, means that the function is expecting an address of a variable of type int. The ‘*’ in variable declaration tells the compiler that the variable is of a special kind, which may be denoted as “a pointer to a variable of type n” where n is the declaration of data type (e.g., int, short, char, etc.). It may simplify it a bit if we replace the word “pointer” with the word “address.” This way, the declaration of the function parameter may be interpreted as:

“counter is a pointer to a variable of type int.”

or, to make it even simpler

“counter is a variable that may contain an address of a variable of type int.”

Alright, it should be understandable by now. If it is not, please feel free to use the comments section below for your remarks.

Assigning values to pointers

As we know by now - pointers are a particular type of variables that may only contain addresses. Therefore, we cannot merely assign them the value we want (in fact we can, but this is out of the scope for this post). Instead, we need to tell the compiler to do so. Thus, if we want to call the increment_counter() function and pass myInteger as a parameter, we do it like this:

increment_counter(&myInteger);

Where the ‘&’ operator tells the compiler to use the address where the myInteger is stored rather than the value which is stored there. Therefore, we may interpret this like:

increment_counter(take the address of the myInteger variable)

or

increment_counter(take the address labeled as myInteger).

Which equivalent in Assembly (regardless of whether the variable is local or global) would be:

lea eax, [myInteger]                      ; Load EAX register with the address of myInteger
push eax                                             ; pass it as a function parameter
call increment_counter                ; call the function

Although I decided to use a function call as an example here, I cannot skip direct assignment of pointers. Let’s introduce another variable called myIntegerPtr which would be a pointer to a variable of type int. Then our declarations would look like this:

int myInteger;
int* myIntegerPtr;

So, just for the sake of the example, let’s use myIntegerPtr as a parameter to be passed to the increment_counter() function. In such case, we need to first assign it the address of myInteger:

myIntegerPtr = &myInteger

Which means:

Take the address of myInteger (or address labeled as myInteger) and store it in the myIntegerPtr variable.

Retrieving values pointed by pointers

Now that we know what pointers are (and they are just variables storing addresses), we still need to know how to access the values they are pointing at. This process is called “dereferencing the pointer.” For the sake of example, let’s examine the body of the increment_counter() function:

int increment_counter(unsigned int* counter)
{
    return ((*counter)++ == 0xffffffff)? 1 : 0;
}

The expression *counter means “take the integer value stored at the address contained in counter.”

Pointer arithmetics

You have definitely noticed the brackets around the *counter, should we omit them, the expression would result in incrementing the pointer (in fact incrementing the address) and reading the value from a not intended location, which may, in certain circumstances, cause the access violation exception.

This brings us to the topic of pointer arithmetics. There is nothing too much special about it, and, although everything may be put in a single sentence, let’s still review an example.

Say we have a string

char s[] = “Hello, World!\n”;

And we want to calculate its length (in fact implement a strlen() function of our own). We, of course, may treat the s as an array of char and check every member for being 0 by trivially indexing into the array. However, for the sake of example and in accordance with our topic, we will use a pointer:

char* cp;

So, cp is a pointer to a variable of type char. The first thing we do – we assign cp with the address of the first char in s:

cp = &s[0];

Of course, we could have written “cp = s,” but we are not going to dive into arrays here. Besides, the above notation is, in my opinion, a bit more demonstrative.

The next steps would be:

1.     Compare the char pointer by cp against 0;
2.   If equals, then break out of the loop;
3.    Otherwise increment the pointer: cp++;
4.    Go to step 1

Still nothing special here, but that is because we were operating on an array of char, which is one byte (usually). Should we have been dealing with an array of shorts or integers, we would still write “cp++,” but the actual effect would differ.

It is essential to understand that adding or subtracting value to/from a pointer, in fact, means adding or subtracting that value times the size of the type.

For example, working with a pointer to an integer:

myIntegerPtr – contains the address of myInteger;

myIntegerPtr++ would make it contain the address of myInteger + sizeof(int) bytes (which, in this case, is 4).

myIntegerPtr += 2 would make it contain the address of myInteger + 2 * sizeof(int) and so on, and so forth.

Pointer to a pointer

Last thing to mention for today – pointer to a pointer, which merely means “a variable containing the address of a variable containing the address of a variable of a certain type.”

We are all familiar with the following construct:

int main(int argc, char** argv)
{
}

where char** argv means “argv is a pointer to a pointer to a variable of type char.” And in fact, argv points to an array of pointers to strings representing command line parameters:

Argv à 

                Param0 à

                Param1 à

                …

                ParamN à

Dereferencing such constructs is not much harder than dereferencing simple pointers. For example:

*argv – gives us Param0

**argv – gives us the char pointed by Param0

*(argv + 1) – gives us Param1

*(*(argv + 1)) – gives us the char pointed by Param1

*(*(argv + 1) + 1) – gives us the char at Param1 + 1

You would hardly ever need to use more than two or, in the worst case, three levels of indirection. If you do, you should probably revise your code. 

P.S. This is about as much as needed for understanding what pointers are and how to use them. Other topics, like, for example, memory management, etc., although involve pointers but are entirely different matters.

Hope this post was helpful.

"Mastering Assembly Programming" - the review

I have recently been contacted by one of my respected readers, who has not only purchased the book but also offered to write a review once he is done reading it. Given the fact that, by now, I have not received much feedback on the book at all, I agreed, although felt a bit uncomfortable. After all, this is my first book and the only reviews I saw this far were those written by the editorial staff and the technical reviewer, getting one from a reader was rather exciting.

For some reasons (still unknown to us) it was not possible to publish the review on publisher's website as the option just disappeared, therefore, I am posting it here having the kind permission to do so from Mr. Tristano Ajmone - the author of the review. 

(The review has been originally posted as a comment on my Google+ page here https://plus.google.com/+AlexeyLyashko/posts/DeBndGe5sgH?cfem=1).

REVIEW OF "MASTERING ASSEMBLY PROGRAMMING"

10/10, because this book lives up to its promises. It's a pragmatic book by a pragmatic author. Lyashko proves himself an expert guide into the land of Assembly language; all along the journey he's always focused on the reader, on how to make his journey of exploration comfortable and profitable.


It's not the usual academic book, and it's not the usal course or manual either: Lyashko never takes the stance of the professor speaking from the high pulpit of academy, and it doesn't take the one-morsel-at-the-time approach either. The experience of this book is not that of speaking about Assembly: it's about entering the land of Assembly and finding a local guide to welcome you.

Like a native who loves his land, Lyashko warmly welcomes you to the esotic land of Assembly and makes you feel comfortably at home right from the onset. He provides you the right equipment for the journey, and dives you head-on into a guided tour of the land. Each trip (or chapter) has a specific goal to it, and as you walk toward it Lyashko shows you the surroundings, explains the history of the places you walk by, the local traditions, the anecdotes; and by time you reach your destination you no longer feel a stranger in a strangeland, you feel at home — and even if you don't yet speak the native language, you start to make sense of its sounds, to see patterns of speech emerge, you begin to reckognize isolated words and sentences. All of a sudden, the journey seems less scarier, it's turning into a thrilling adventure, and the dream of becoming a citizen of Assembly land seems an achievable goal.

The leap from high-level languages to Assembly is a huge one. No single book could ever bridge fully across this gap. At the mother-board level there are no alibis: you can't abstract away from the building blocks of the machinery. It's a very thin edge for a student to walk over, and for the authors the risk is always between extermes: providing to many details or to little, too much theory or to little. It takes a novel approach to solve these ancient dilemmas, and I believe this book does an excellent job at providing the right balance of theory and practice. But most of all, it takes a certain type of author too.

I belong to the old school, and I strongly believe that what really counts in programming is the method — nothing but the method. This book gave me method: where all I could see before where technical data and isolated tools scattered on the workbench, now I've seen a good example of how the master craftsman employs these tools in the art.

As an avid reader, I've come across too many books where competent authors failed to relate directly to their readers. The subject at matter is never as important as the gap that separates the reader from the subject itself. Programming books are always written for competent people (and it would be offensive to suggest otherwise), so it's never an issue of having to teach again the alphabet from scratch to reader; it's about filling the gap between what the reader already knows and how his knowledge has to adpat and relate to the new subject at hand. Lyashko is clearly a good teacher, he knows what Assembly newbies need in order to jump on the running wagon and get going; he knows the right balances and measures required to feed the reader's desire to learn and at the same time keep his appetite alive — avoiding an indegestion of theory, or tech-jargon poisoning.

At the end of the day, every introductiory book to Assembly has to cover the same topics (and this book is no execption). What makes the difference between a good and ejoyable book and a boring and over-complex text is the author's skills as a teacher. And I'm truely impressed by Alexey Lyashko's pedagogy: it's fresh, innovative, pragmatic, and speaks to the reader as peers do: by putting himself in the reader's shoes, he's able to relate at the same level of the reader, and raise him up all the way to the master's level — which is the promise of this book.

We all have to stand on the shoulders of giants to encompass new horizons. "Mastering Assembly Programming" gently lifts us from the barren grounds and places us on the giant's shoulders, allowing us the see the promised land of Assembly with our own eyes. Like any other journey, teachers can only take us so far by hand, then we have to carry on with our own legs — to expect more would be a mistake on the student's side, to expect less would be an unforgivable mistake on the teacher's side. Lyashko is guilty of none, for he has fulfilled his promise to the reader.

Tristano Ajmone (Italy)

Faking KERNEL32.DLL - an Amateur Sandbox

As a part of my work (read "fun") of maintaining this blog, I am constantly checking the statistic information on traffic sources and keywords (it's nice to know that people are getting here via Google) in order to see whether my readers are getting what they are looking for (personally, I see no reason in simply "streaming my consciousness to the masses" as this is not the point of this blog). Sometimes, it gives an idea of what is missing but still related to system and low level programming.

A couple of days ago, I saw that someone was looking for a way to load and use fake KERNEL32.dll and I realized that this information has not yet been covered here. There is no source code for this article as I am a bit short on time to write it, but I will do my best to provide as much information as possible so, those who would want to try it would have no problem doing that.

KERNEL32.DLL

First notable thing about KERNEL32.dll is that it is always loaded, regardless of whether a running executable imports anything from it (this is briefly covered here). Same as NTDLL.dll (well, KERNEL32.dll imports from it). This library provides interfaces for interaction with deeper levels of "user land" part of the operating system for the running executable and some of other dynamic link libraries loaded into process' memory. 

Knowing all that, the first thought may be: "how are we going to fake it if all the rest depends on it?". The solution is easier than one could think at first. However, we should keep in mind, that some programs may import from NTDLL.dll directly, bypassing the KERNEL32.dll (which used to happen quite often in the world of malware), meaning that once you faked KERNEL32.dll, you may have to fake NTDLL,as well.

Preparations

We should start with writing a good old simple DLL/code injector. It is easier to dissect the victim process from inside. This is the simplest part and it is covered in this and this posts of this blog. Roughly saying, the injector should be able to create a victim process in suspended state by passing the CREATE_SUSPENDED process creation flag to CreateProcess API.

Writing the code or the DLL we are going to inject is a harder task as this code is intended to perform the tasks described below in order of execution.

Load Fake KERNEL32.dll

Let's assume, that we already have a ready to use fake KERNEL32.dll (we'll get back to creation of fake dll a bit later). This is quite simple - call LoadLibrary function from your code. One thing worth mentioning is that MSDN is not suggesting to use LoadLibrary in your DllMain function. Therefore, if you decide to use DLL Injection instead of code injection, then better use the approach described in "Advanced DLL Injection" article. 

Fake KERNEL32.dll should simply import all API's from the original one. Don't be mistaken - import, not forward it's exports at least as long as we are talking about API functions, but you may safely forward exported objects and variables to the original one.

Resolve Victim's Imports

By the time we get our code/DLL running inside the suspended victim process, all of it's imports should already have been resolved. What we still have to do, is to replace all API addresses exported from the original KERNEL32.dll with corresponding addresses in our fake one.

Here is a link to Microsoft's specifications of MS PE and MS COFF file formats - would be useful digging through imports and export.

Hide the Original KERNEL32.dll

While performing the aforementioned actions may be enough in case of a regular application, we should take some precautions in case of malicious code. My suggestion is to hide the original KERNEL32.dll by replacing its entry in the list of LDR_MODULE structures in PEB with the one describing our fake KERNEL32.dll, just like we would hide an injected DLL in the "Hiding Injected DLL in Windows" article.

Creation of Fake KERNEL32.dll

This may sound scary, but there is no need to worry (at least not too much). All that we need in order to create one, is a C compiler (or whatever high level language you prefer) and any assembler (I use FASM as usual).

Dump KERNEL32.dll to ASM Source

No, of course we do not have to disassemble the whole DLL and dump it to a corresponding Assembly source. Instead, what we have to do, is write a small application in high level language (you may try to do it in Assembly if you want) that would parse the export table of the original KERNEL32.dll and create a set of Assembly source files: one for code, one for data (if needed), one for import and one for export sections.

Want it or not, but the application has to generate a bit of Assembly code for at least transferring the execution flow to an API function in the original KERNEL32.dll. For example, if we have no interest in, let's say, ExitProcess, then our fake ExitProcess should look similar to this:

fake_ExitProcess:

   ; As we are not tracing/logging this function, we simply let the

   ; original ExitProcess shoot

   jmp dword [real_ExitProcess]

However, the code would be different for APIs of interest. For example, the CreateFileA API would be implemented like this:

fake_CreateFileA:

   ; We pass control to a fake CreateFileA, which is implemented in

   ; a separate DLL imported by our fake KERNEL32.dll

   ; Parameters are already on the stack, so we simply jump.

   ; Don't forget to declare the fake function as STDCALL 

   ; (remember #define WINAPI __declspec(stdcall) ? )

   jmp dword [our_CreateFileA]

The Assembly source file containing code for the import section would then contain the following:

section '.idata' import data readable writable

   library original, 'kernel32.dll',\            ;Import original KERNEL32.dll

           fake,     'our_dll_with_fake_api.dll' ;Import a DLL with fake APIs

   import original,\

      real_ExitProcess, 'ExitProcess'

   import fake,\

      our_CreateFileA, 'whatever you call it here'

Now, finally, we get to the export section's code:

section '.edata' export data readable

   export 'KERNEL32.dll',\              ;FASM does not care about what you type here, 

                                        ;so let's be fake to the end and pretend 

                                        ;to be KERNEL32.dll

      fake_ExitProcess, 'ExitProcess',\

      fake_CreateFileA, 'CreateFileA'

Finally the main source file, the one that would assemble all the rest together:

format PE DLL at 0x10000000

include 'code.asm'

include 'import.asm'

include 'export.asm'

section '.reloc' fixups data discardable

compile it with FASM and you have your fake KERNEL32.dll.

Implementation of Fake API

As it has been mentioned above, there are some functions we would want to trace. Those should have some custom implementation, preferably in a separate DLL (which would be loaded by Windows loader at the time it resolves our fake KERNEL32.dll's dependencies). Below is a diagram of the interactions between all the modules:

Interactions between modules involved in faking.

And here is an example of such fake API:

HANDLE WINAPI fake_CreateFileA(

                               LPCSTR lpFileName,

                               DWORD dwDesiredAccess,

                               DWORD dwShareMode,

                               LPSECURITY_ATTRIBUTES lpSA,

                               DWORD dwCreationDisposition,

                               DWORD dwFlagsAndAttributes,

                               HANDLE hTemplateFile)

{

   fprintf(log_file, "CreateFileA(list params here)\n", params);

   return CreateFileA(lpFileName, 

                      dwDesiredAccess, 

                      dwShareMode, 

                      lpSA, 

                      dwCreationDisposition,

                      dwFlagsAndAttributes,

                      hTemplateFile);

}

Of course, you may implement addition mechanisms within this DLL, e.g. let it communicate with another application via sockets or pipes, but this is deserves a separate article.

P.S. 

My personal suggestion is to insert more code into each function inside the fake KERNEL32.dll so that it would look more realistic to the victim process (should it try to do anything with it).

Hope this article was useful. 

See you at the next.

Trivial Artificial Neural Network in Assembly Language

Source code for this article may be found here.

Note for nerds: The code shown in this article may be incomplete and may not contain all the security checks you would usually perform in your code as it is given here for demonstration purposes only. Downloadable source code may contain bugs (there is no software without bugs at all). It is provided as is without any warranty. You may use and redistribute it as you wish while mentioning this site and the author.

I was recently digging through my sources and came across a small ANN (artificial neural network) library I wrote several months ago in 64 bit Intel Assembly language (FASM syntax) and decided to share it with my respected readers hoping that it may be useful in some cases.

Artificial Neural Network

Internet is full of articles covering this topic either in general or in depth. Personally, I would prefer not to create yet another clone with pictures of Synapses, etc. In short ANN is a computational model inspired by the way our brain seems to work. There is a good article on Wikipedia giving quite a good amount of explanations. It seems to be important to mention, that saying "ANN" people usually think of Perceptron or Multilayer Perceptron, but there are much more types out there. You should check out this article if you are interested. 

However, this article covers implementation of Multilayer Perceptron in Assembly language, which appears to be easier than it sounds. The library is appropriate for creation of multilayer perceptrons with any number of hidden layers, any number of input and output neurons, although, it is bound to 64 bit Linux, I will try to explain how you can change the code to make it compatible with 64 Windows, but it would take much more effort to actually rewrite the whole thing to run on 32 bit platforms.

Neuron

This is the basis of the whole project. Neuron is the main part of the calculation. In this example, all neurons are arranged into a linked list, having input neurons at the beginning of the list and output neurons at its end. It is important to mention that they would all be processed in the same order they appear in the linked list. First of all, let's define a structure, that would contain all the information we need for a single neuron:

struc list

{

   .prev_ptr    dq ?

   .next_ptr    dq ?

}

struc neuron

{

   .list        list  ;Pointers to previous and next neurons

   .input       dq ?  ;Pointer to the first input synapse

   .output      dq ?  ;Pointer to the first output synapse

   .value       dq ?  ;Resulting value of the neuron

   .signal      dq ?  ;Error signal

   .sum         dq ?  ;Sum of all weighted inputs

   .bias        dq ?  ;Bias weight (threshold)

   .bias_delta  dq ?  ;Bias weight delta

   .index       dw ?  ;Index of the given neuron

   .num_inputs  dw ?  ;Number of input synapses

   .num_outputs dw ?  ;Number of output synapses

   .type        dw ?  ;Type of the neuron (bit field)

   .size        = $ - .list

}

Figure 1 shows the arrangement of neurons in a perceptron designed to perform XOR operation. It has 2 input neurons, three neurons in the hidden layer and two output neurons. Arrows show the order of processing.

Figure 1

I implemented this perceptron with 2 output neurons for testing purposes only, as it could well be implemented with a single output neuron, where output value > 0.5 would be 1 and below would be 0.

Synapse

There would be no perceptron without synaptic links. This is where the following structure appears on the scene.

struc synaps
{
   .inputs        list   ;Pointers to previous and next input synapses
                         ;if such exist
   .outputs       list   ;Pointers to previous and next output synapses
                         ;if such exist
   .value         dq ?   ;Value to be transmitted
   .weight        dq ?   ;Weight of the synapse
   .delta         dq ?   ;Weight delta
   .signal        dq ?   ;Error signal
   .input_index   dw ?   ;Index of the input neuron in the list of neurons
   .output_index  dw ?   ;Index of the output neuron in the list of neurons
                  dd ?   ;Alignment
   .size          = $ - .inputs
}

At first, it may be a bit hard to understand why there are so many pointers in both structures. Unfortunately, my verbal abilities are far from being perfect (especially, given that English is not my mother tongue), therefore, let me illustrate the way neurons are interconnected with synapses in this implementation first (in a hope that my graphic abilities are not worse then verbal).

Figure 2

Figure 2 shows that each neuron (except the output ones) has a pointer (neuron.output) to a list of synapses that need to be fed with this neuron's calculated value. For a neuron, its output synapses are linked with synaps.outputs pointers. In turn, each neuron (except the input ones) has a pointer (neuron.input) to a list of synapses to collect inputs from. On the figure, each gray arrow goes from a neuron in the left layer to a neuron in the right layer through the corresponding synaptic link.

Processing a Single Neuron

Each neuron in the network is processed with the same function which prototype in C is like this:

void neuron_process(neuron_t* n, int activation_type);

where n is a pointer to the neuron we want to process and activation_type specifies which activation function should be used. As I have mentioned above, this implementation only has one activation function - logistic (aka exponential):

f(x) = 1.0 / (1.0 + exp(-2 * x))

The following piece of code is an Assembly implementation of EXP():

;double exp(double d)

exp:

   push   rbp

   mov    rbp, esp

   sub    rsp, 8

   push   rbx rcx rdx rdi rsi

   movsd  qword [rbp-8], xmm0

   fld    qword [rbp-8]

   fld2e

   fmulp  st1, st0

   fld    st0

   frndint

   fsub   st1, st0

   fxch   st1

   f2xm1

   fld1

   faddp  st1, st0

   fscale

   fstp   st1

   fstp   qword [rbp-8]

   fwait

   movsd  xmm0, qword [rbp-8]

   pop    rsi rdi rdx rcx rbx

   add    rsp, 8

   leave

   ret

Now the x itself. x is a sum of products of value and weight of all input synaptic links plus bias weight of a neuron. The result of f() is then stored to each and every output synaptic link (if not output neuron) in accordance to the diagram shown on figure 3:

Figure 3

.

The Net

We are almost done with building out net. Let's define a structure that would incorporate all the information about our perceptron and all values needed for training and execution:

struc net

{

   .neurons     dq ? ;Pointer to the first neuron in the linked list of neurons

   .outs         dq ? ;Pointer to the first output neuron

   .num_neurons  dd ? ;Total amount of neurons

   .activation   dd ? ;Which activation method to use (we only have one here)

   .qerror       dq ? ;Mean quadratic error

   .num_inputs   dw ? ;Number of input neurons

   .num_outputs  dw ? ;Number of output neurons

   .rate         dq ? ;Learning rate regulates learning speed

   .momentum     dq ? ;Roughly saying - error tolerance

   .size         = $ - .neurons

}

   

The Fun

The source code attached to this article implements all the functions needed to manipulate the network as needed. All functions are exported by the library and described in "ann.h". However, we only need to deal with few of them:

net_t* net_alloc(void);

This function allocates  the net_t object and returns a pointer.

void net_fill(net_t* net, int number_of_neurons, int number_of_inputs, int number_of_outputs);

This function populates the net with requested amount of neurons and sets all values and pointers accordingly.

void net_set_links(net_t* net, int* links);

This function is responsible for setting up all the synaptic links between neurons. While net is a pointer to previously allocated net_t structure, links is a pointer to the array of integer pairs terminated by a pair of 0's:

int pairs[][2]={

   {1, 3},

   {1, 4},

   {2, 4},

   {2, 5},

   {3, 6},

   {3, 7},

   {4, 6},

   {4, 7},

   {5, 6},

   {5, 7},

   {0, 0}};

The above array is exactly the one used in the supplied test application in order to set up links as shown on figure 3.

double net_train(net_t* net, double* values, double* targets);

This function is responsible for everything that is needed in order to train our perceptron using the back-propagation training paradigm. Returns mean quadratic error of all output neurons (which is also accessible through net->qerror).

values - array of values to be fed into the net prior to running it (the function does not check whether the length of the array is appropriate, so your code is responsible for that);

targets - array of expected results.

Due to the fact that we are using logistic activation function, it is necessary to normalize the input data to be in the range [0 < x < 1] (outputs would be in the same range as well).

Run this function as many times as needed in order to get a reasonable error. You will have to "play" with rate and momentum parameters to get best values, but you may start with 0.9 for rate and 0.02 for momentum. It is important to specify those values as the library does not check whether they are set or not!

void net_run(net_t* net, double* values);

This function is used in order to run the net. 

values - same as in case of net_train function;

This function does not return a value, so you have to manually access net->outs.

Attached Source Code

The attached source code may be compiled with flat assembler:

   fasm libann.asm libann.o

and linked to any C code compiled with GCC on 64 bit Linux.

Making the Code Windows Compatible

This requires a piece of work. First of all, you would have to edit the libann.asm file, changing the format ELF64 to format MS64 COFF and sections' attributes accordingly. You would also have to make some changes to the code. 64 bit Linux uses AMD64 ABI, while Microsoft has its own. Major differences are in how parameters are passed to functions. While in Linux they are passed via RDI, RSI, RDX, RCX, R8 and R9 (all the rest on the stack in reverse order) registers for integers and XMM0 - XMM7 for doubles, Microsoft uses RCX, RDX, R8 and R9 for integers and XMM0 - XMM3 for doubles and any additional parameters are passed on stack in reverse order.

Output

The output for XOR problem should look similar to this:

Output

Thanks for reading! Hope this article was interesting and may be even helpful.

See you at the next!