Gene-Ontologies and Annotation

Gene-Ontology Analysis

In the early days of microarray analysis, people were happy if they got a handful of differentially-expressed genes that they could validate or follow-up. However, with later technologies (and depending on the experimental setup) we might have thousands of statistically-significant results, which no-one has the time to follow-up. Also, we might be interested in pathways / mechanisms that are altered and not just individual genes.

In this section we move towards discovering if our results are biologically significant. Are the genes that we have picked statistical flukes, or are there some commonalities.

There are two different approaches one might use, and we will cover the theory behind both

Theory Part I: Over-representation analysis

  • “Threshold-based”: require defintion of a statistical threshold to define list of genes to test (e.g. FDR < 0.01)
  • Hypergeometric test or Fisher’s Exact test generally used.

The question we are asking here is;

“Are the number of DE genes associated with Theme X significantly greater than what we might expect by chance alone?”

Where Theme X could be genes belonging to a particular GO (Gene Onotology) term.

Let’s imagine that we have a bag full of balls. Each balls represents a gene in the gene universe. - Paint the balls representing our selected list grey, and paint the rest red.

In this small example, we can define;

  • Total number of balls: 40
  • Total number of interesting (grey) balls: 10

Now, lets select a number (say, 12) of the balls at random without seeing into the bag and look at what we get

We have picked, at random, 8 grey balls. Using simulations, we can repeat the process and look at how many grey we get.

The distribution of the data shows what the most-likely values are

We can count how many times each value is observed

trials
   4    5    6    7    8    9   10   11   12 
   1    3   73  403 1333 2638 3168 1922  459 

Dividing by the number of trials gives a probability of sorts

trials
     4      5      6      7      8      9     10     11     12 
0.0001 0.0003 0.0073 0.0403 0.1333 0.2638 0.3168 0.1922 0.0459 

The probability of getting at least a certain number can also be computed

     4      5      6      7      8      9     10     11     12 
0.0001 0.0004 0.0077 0.0480 0.1813 0.4451 0.7619 0.9541 1.0000 
     4      5      6      7      8      9     10     11     12 
0.9999 0.9996 0.9923 0.9520 0.8187 0.5549 0.2381 0.0459 0.0000 

Back to our example, the distribution of balls can be expressed as a contingency table, on which we can use a Fisher’s exact test

Total grey balls: 10 Total in subset: 12

The formula for Fishers exact test is;

\[ p = \frac{\binom{a + b}{a}\binom{c +d}{c}}{\binom{n}{a +c}} = \frac{(a+b)!(c+d)!(a+c)!(b+d)!}{a!b!c!d!n!} \]

or less formally;

P = (ways of choosing grey balls) X (ways of non-grey balls amongst subset) / ways of choosing subset

Be careful of how you define the universe

What genes were candidates for selection as universe?

  • All possible genes
  • All genes represented on the chip
  • All genes that have a GO annotation
  • All genes from the chip that pass a non-specific filter

Just changing the size of the universe alone can have a massive effect on the p-value

  • In the formula, we have to divide by the universe size
    • so the probability will become very small as this increases
    • even if all the other numbers stay the same

Worked Example: Drosophila genes regulated by pasilla

In order to get a larger set of differentially-expressed genes across the whole genome, we will use a different dataset where gene counts have already been computed

Preamble

In the study of Brooks et al. 2011, the Pasilla (PS) gene, Drosophila homologue of the Human splicing regulators Nova-1 and Nova-2 Proteins, was depleted in Drosophila melanogaster by RNAi. The authors wanted to identify exons that are regulated by Pasilla gene using RNA sequencing data. Total RNA was isolated and used for preparing either single-end or paired-end RNA-seq libraries for treated (PS depleted) samples and untreated samples. These libraries were sequenced to obtain a collection of RNA sequencing reads for each sample. The effects of Pasilla gene depletion on splicing events can then be analyzed by comparison of RNA sequencing data of the treated (PS depleted) and the untreated samples. The genome of Drosophila melanogaster is known and assembled. It can be used as reference genome to ease this analysis. In a reference based RNA-seq data analysis, the reads are aligned (or mapped) against a reference genome, Drosophila melanogaster here, to significantly improve the ability to reconstruct transcripts and then identify differences of expression between several conditions.

Data

To save time and memory, pre-counted files are available online (https://zenodo.org/record/1185122#.WqGfbnW0PCI)

  • GSM461176_untreat_single.counts
  • GSM461177_untreat_paired.counts
  • GSM461178_untreat_paired.counts
  • GSM461179_treat_single.counts
  • GSM461180_treat_paired.counts
  • GSM461181_treat_paired.counts
  • GSM461182_untreat_single.counts

Exercise

  • Download the files from the zenodo link, and upload them to Galaxy
  • Use DESeq2 as with the previous section
  • Filter the results using the condition c7 < 0.05 and (c3 > 1.0 or c3 < -1.0)

(Optional) Sort by a column:

We can sort the table by values in a particular column (eg. log fold change)

  1. Filter and Sort -> Sort
  2. Select the most recent filtered table as the dataset
  3. Sort log fold change by Descending order

Preparing the data for GOrilla

We will be using the online tool GOrilla

  1. Select the table that you have filtered and sorted
  2. Operation: Keep
  3. Select only column 1 (Gene IDs) from List of Fields
  4. Save the outputted dataset as “DE_Genes.txt”

Make a background list of genes:

This is the list of all the genes you tested in your RNAseq pipeline.

  1. Filter and Sort -> Filter
  2. Select the DESeq2 results from the previous step prior to p-value and fold-change filtering
  3. Text Manipulation -> Cut
  4. Keep the first column
  5. Download and save the resulting file of gene names as “Background.txt”

Using GOrilla

  1. Go to http://cbl-gorilla.cs.technion.ac.il/
  2. Read the “Running Example”
  1. Choose Organism: Drosophila melanogaster
  2. Choose running mode: Two unranked lists of genes
  3. Upload DE_Genes.txt as the Target set.
  4. Upload Background.txt as the Background set.
  5. Choose an Ontology: Process
  6. Search Enriched GO terms

Theory Part II: Threshold-free

For these tests, we don’t have to specify a statistical threshold and use the test statistics (log fold-change) from all genes as the input to the test. The popular Gene Set Enrichment Analysis (GSEA) uses this approach. These tests can be used to detect differential expression for a group of genes, even when the effects are too small or there is too little data to detect the genes individually.

gsea

gsea

Subramanian et al, PNAS 2005

The Broad institute provides a version of GSEA that can be run via a java application. Another option is GeneTrail which has the option to do a GSEA-style analysis.

Preparing data for GeneTrail

  1. Text Maniupulation -> Filter with condition c3 != 'NA'
  • select DESeq2 result file... as input
  1. Text Manipulation -> Cut
  • selecting columns c1 c3 as the columns to cut
  1. Download the filtered, cut, table as a file (e.g. gsea-input.txt)

Running GeneTrail

Go to the GeneTrail website

  1. Under Transcriptome, select Start Analysis
  2. Under Upload Data select the data source to be a text file and browse to where you downloaded the file gsea-input.txt
  3. Keep the default categories and Start Analysis
  4. Explore the results to see what pathways are reported
LS0tCnRpdGxlOiAiT250b2xvZ2llcyBhbmQgRW5yaWNobWVudCIKYXV0aG9yOiAiTWFyayBEdW5uaW5nIgpkYXRlOiAnYHIgZm9ybWF0KFN5cy50aW1lKCksICJMYXN0IG1vZGlmaWVkOiAlZCAlYiAlWSIpYCcKb3V0cHV0OiAKICBodG1sX25vdGVib29rOiAKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwotLS0KCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IEZBTFNFKQpgYGAKCgpgYGB7ciBlY2hvPUZBTFNFLG1lc3NhZ2U9RkFMU0V9CmxpYnJhcnkobGltbWEpCmBgYAoKIyBHZW5lLU9udG9sb2dpZXMgYW5kIEFubm90YXRpb24KCiMjIEdlbmUtT250b2xvZ3kgQW5hbHlzaXMKCkluIHRoZSBlYXJseSBkYXlzIG9mIG1pY3JvYXJyYXkgYW5hbHlzaXMsIHBlb3BsZSB3ZXJlIGhhcHB5IGlmIHRoZXkgZ290IGEgaGFuZGZ1bCBvZiBkaWZmZXJlbnRpYWxseS1leHByZXNzZWQgZ2VuZXMgdGhhdCB0aGV5IGNvdWxkIHZhbGlkYXRlIG9yIGZvbGxvdy11cC4gSG93ZXZlciwgd2l0aCBsYXRlciB0ZWNobm9sb2dpZXMgKGFuZCBkZXBlbmRpbmcgb24gdGhlIGV4cGVyaW1lbnRhbCBzZXR1cCkgd2UgbWlnaHQgaGF2ZSB0aG91c2FuZHMgb2Ygc3RhdGlzdGljYWxseS1zaWduaWZpY2FudCByZXN1bHRzLCB3aGljaCBuby1vbmUgaGFzIHRoZSB0aW1lIHRvIGZvbGxvdy11cC4gQWxzbywgd2UgbWlnaHQgYmUgaW50ZXJlc3RlZCBpbiBwYXRod2F5cyAvIG1lY2hhbmlzbXMgdGhhdCBhcmUgYWx0ZXJlZCBhbmQgbm90IGp1c3QgaW5kaXZpZHVhbCBnZW5lcy4KCkluIHRoaXMgc2VjdGlvbiB3ZSBtb3ZlIHRvd2FyZHMgZGlzY292ZXJpbmcgaWYgb3VyIHJlc3VsdHMgYXJlICoqKmJpb2xvZ2ljYWxseSBzaWduaWZpY2FudCoqKi4gQXJlIHRoZSBnZW5lcyB0aGF0IHdlIGhhdmUgcGlja2VkIHN0YXRpc3RpY2FsIGZsdWtlcywgb3IgYXJlIHRoZXJlIHNvbWUgY29tbW9uYWxpdGllcy4gCgpUaGVyZSBhcmUgdHdvIGRpZmZlcmVudCBhcHByb2FjaGVzIG9uZSBtaWdodCB1c2UsIGFuZCB3ZSB3aWxsIGNvdmVyIHRoZSB0aGVvcnkgYmVoaW5kIGJvdGgKCgogICAgCgojIyBUaGVvcnkgUGFydCBJOiBPdmVyLXJlcHJlc2VudGF0aW9uIGFuYWx5c2lzCgotICJUaHJlc2hvbGQtYmFzZWQiOiByZXF1aXJlIGRlZmludGlvbiBvZiBhIHN0YXRpc3RpY2FsIHRocmVzaG9sZCB0byBkZWZpbmUgbGlzdCBvZiBnZW5lcyB0byB0ZXN0IChlLmcuIEZEUiA8IDAuMDEpCi0gSHlwZXJnZW9tZXRyaWMgdGVzdCBvciBGaXNoZXIncyBFeGFjdCB0ZXN0IGdlbmVyYWxseSB1c2VkLgoKVGhlIHF1ZXN0aW9uIHdlIGFyZSBhc2tpbmcgaGVyZSBpczsKCj4gKioqIkFyZSB0aGUgbnVtYmVyIG9mIERFIGdlbmVzIGFzc29jaWF0ZWQgd2l0aCBUaGVtZSBYIHNpZ25pZmljYW50bHkgZ3JlYXRlciB0aGFuIHdoYXQgd2UgbWlnaHQgZXhwZWN0IGJ5IGNoYW5jZSBhbG9uZT8iKioqCgpXaGVyZSBUaGVtZSBYIGNvdWxkIGJlIGdlbmVzIGJlbG9uZ2luZyB0byBhIHBhcnRpY3VsYXIgR08gKEdlbmUgT25vdG9sb2d5KSB0ZXJtLgoKTGV0J3MgaW1hZ2luZSB0aGF0IHdlIGhhdmUgYSBiYWcgZnVsbCBvZiBiYWxscy4gRWFjaCBiYWxscyByZXByZXNlbnRzIGEgZ2VuZSBpbiB0aGUgKmdlbmUgdW5pdmVyc2UqLiAKLSBQYWludCB0aGUgYmFsbHMgcmVwcmVzZW50aW5nIG91ciBzZWxlY3RlZCBsaXN0IGdyZXksIGFuZCBwYWludCB0aGUgcmVzdCByZWQuCgoKIVtdKGltYWdlcy9iYWctYW5kLWJhbGxzLnBuZykKCkluIHRoaXMgc21hbGwgZXhhbXBsZSwgd2UgY2FuIGRlZmluZTsKCi0gVG90YWwgbnVtYmVyIG9mIGJhbGxzOiA0MAotIFRvdGFsIG51bWJlciBvZiBpbnRlcmVzdGluZyAoZ3JleSkgYmFsbHM6IDEwCgpOb3csIGxldHMgc2VsZWN0IGEgbnVtYmVyIChzYXksIDEyKSBvZiB0aGUgYmFsbHMgYXQgcmFuZG9tIHdpdGhvdXQgc2VlaW5nIGludG8gdGhlIGJhZyBhbmQgbG9vayBhdCB3aGF0IHdlIGdldAoKIVtdKGltYWdlcy9waWNrZWQtYmFsbHMucG5nKQoKCldlIGhhdmUgcGlja2VkLCBhdCByYW5kb20sIDggZ3JleSBiYWxscy4gVXNpbmcgc2ltdWxhdGlvbnMsIHdlIGNhbiByZXBlYXQgdGhlIHByb2Nlc3MgYW5kIGxvb2sgYXQgaG93IG1hbnkgZ3JleSB3ZSBnZXQuIAoKVGhlIGRpc3RyaWJ1dGlvbiBvZiB0aGUgZGF0YSBzaG93cyB3aGF0IHRoZSBtb3N0LWxpa2VseSB2YWx1ZXMgYXJlCgpgYGB7cn0KI3NlZSA/cmh5cGVyIGZvciBhcmd1bWVudCBkZWZpbml0aW9uCnRyaWFscyA8LSByaHlwZXIoMTAwMDAsNDAsMTAsMTIpCmhpc3QodHJpYWxzKQpgYGAKCldlIGNhbiBjb3VudCBob3cgbWFueSB0aW1lcyBlYWNoIHZhbHVlIGlzIG9ic2VydmVkCgpgYGB7cn0KdGFibGUodHJpYWxzKQpgYGAKCkRpdmlkaW5nIGJ5IHRoZSBudW1iZXIgb2YgdHJpYWxzIGdpdmVzIGEgcHJvYmFiaWxpdHkgb2Ygc29ydHMKCmBgYHtyfQp0YWJsZSh0cmlhbHMpLzEwMDAwCgpgYGAKClRoZSBwcm9iYWJpbGl0eSBvZiBnZXR0aW5nICphdCBsZWFzdCogYSBjZXJ0YWluIG51bWJlciBjYW4gYWxzbyBiZSBjb21wdXRlZAoKYGBge3J9CmN1bXN1bSh0YWJsZSh0cmlhbHMpLzEwMDAwKQoxLWN1bXN1bSh0YWJsZSh0cmlhbHMpLzEwMDAwKQpgYGAKCgoKQmFjayB0byBvdXIgZXhhbXBsZSwgdGhlIGRpc3RyaWJ1dGlvbiBvZiBiYWxscyBjYW4gYmUgZXhwcmVzc2VkIGFzIGEgY29udGluZ2VuY3kgdGFibGUsIG9uIHdoaWNoIHdlIGNhbiB1c2UgYSBGaXNoZXIncyBleGFjdCB0ZXN0CgpUb3RhbCBncmV5IGJhbGxzOiAxMApUb3RhbCBpbiBzdWJzZXQ6IDEyCgpgYGB7cn0KZGYgPC0gZGF0YS5mcmFtZShTZWxlY3Rpb25faW4gPSBjKDgsNCksIFNlbGVjdGlvbl9vdXQgPSBjKDIsMjYpKQpyb3duYW1lcyhkZikgPC0gYygiR3JleV9pbiIsICJHcmV5X291dCIpCmRmCmBgYAoKCgpgYGB7cn0KZGYgPC0gZGF0YS5mcmFtZShTZWxlY3Rpb25faW4gPSBjKCJhIiwiYyIsImErYyIpLCBTZWxlY3Rpb25fb3V0ID0gYygiYiIsImQiLCJiK2QiKSwgUm93VG90YWwgPSBjKCJhICtiIiwiYytkIiwiYStiK2MrZCAoPW4pIikpCnJvd25hbWVzKGRmKSA8LSBjKCJHcmV5X2luIiwgIkdyZXlfb3V0IiwiQ29sdW1uIFRvdGFsIikKZGYKYGBgCgpUaGUgZm9ybXVsYSBmb3IgRmlzaGVycyBleGFjdCB0ZXN0IGlzOwoKJCQgcCA9IFxmcmFje1xiaW5vbXthICsgYn17YX1cYmlub217YyArZH17Y319e1xiaW5vbXtufXthICtjfX0gPSBcZnJhY3soYStiKSEoYytkKSEoYStjKSEoYitkKSF9e2EhYiFjIWQhbiF9ICQkCgpvciBsZXNzIGZvcm1hbGx5OwoKKlAgPSAod2F5cyBvZiBjaG9vc2luZyBncmV5IGJhbGxzKSBYICh3YXlzIG9mIG5vbi1ncmV5IGJhbGxzIGFtb25nc3Qgc3Vic2V0KSAvIHdheXMgb2YgY2hvb3Npbmcgc3Vic2V0KgoKCioqQmUgY2FyZWZ1bCBvZiBob3cgeW91IGRlZmluZSB0aGUgdW5pdmVyc2UqKgoKV2hhdCBnZW5lcyB3ZXJlIGNhbmRpZGF0ZXMgZm9yIHNlbGVjdGlvbiBhcyB1bml2ZXJzZT8KCi0gQWxsIHBvc3NpYmxlIGdlbmVzCi0gQWxsIGdlbmVzIHJlcHJlc2VudGVkIG9uIHRoZSBjaGlwCi0gQWxsIGdlbmVzIHRoYXQgaGF2ZSBhIEdPIGFubm90YXRpb24KLSBBbGwgZ2VuZXMgZnJvbSB0aGUgY2hpcCB0aGF0IHBhc3MgYSAqKm5vbi1zcGVjaWZpYyoqIGZpbHRlcgoKSnVzdCBjaGFuZ2luZyB0aGUgc2l6ZSBvZiB0aGUgdW5pdmVyc2UgYWxvbmUgY2FuIGhhdmUgYSBtYXNzaXZlIGVmZmVjdCBvbiB0aGUgcC12YWx1ZQoKLSBJbiB0aGUgZm9ybXVsYSwgd2UgaGF2ZSB0byBkaXZpZGUgYnkgdGhlIHVuaXZlcnNlIHNpemUKICAgICsgc28gdGhlIHByb2JhYmlsaXR5IHdpbGwgYmVjb21lIHZlcnkgc21hbGwgYXMgdGhpcyBpbmNyZWFzZXMKICAgICsgZXZlbiBpZiBhbGwgdGhlIG90aGVyIG51bWJlcnMgc3RheSB0aGUgc2FtZQoKCiMjIFdvcmtlZCBFeGFtcGxlOiBEcm9zb3BoaWxhIGdlbmVzIHJlZ3VsYXRlZCBieSBwYXNpbGxhCgpJbiBvcmRlciB0byBnZXQgYSBsYXJnZXIgc2V0IG9mIGRpZmZlcmVudGlhbGx5LWV4cHJlc3NlZCBnZW5lcyBhY3Jvc3MgdGhlIHdob2xlIGdlbm9tZSwgd2Ugd2lsbCB1c2UgYSBkaWZmZXJlbnQgZGF0YXNldCB3aGVyZSBnZW5lIGNvdW50cyBoYXZlIGFscmVhZHkgYmVlbiBjb21wdXRlZCAKCiMjIFByZWFtYmxlCgo+IEluIHRoZSBzdHVkeSBvZiBCcm9va3MgZXQgYWwuIDIwMTEsIHRoZSBQYXNpbGxhIChQUykgZ2VuZSwgRHJvc29waGlsYSBob21vbG9ndWUgb2YgdGhlIEh1bWFuIHNwbGljaW5nIHJlZ3VsYXRvcnMgTm92YS0xIGFuZCBOb3ZhLTIgUHJvdGVpbnMsIHdhcyBkZXBsZXRlZCBpbiBEcm9zb3BoaWxhIG1lbGFub2dhc3RlciBieSBSTkFpLiBUaGUgYXV0aG9ycyB3YW50ZWQgdG8gaWRlbnRpZnkgZXhvbnMgdGhhdCBhcmUgcmVndWxhdGVkIGJ5IFBhc2lsbGEgZ2VuZSB1c2luZyBSTkEgc2VxdWVuY2luZyBkYXRhLgpUb3RhbCBSTkEgd2FzIGlzb2xhdGVkIGFuZCB1c2VkIGZvciBwcmVwYXJpbmcgZWl0aGVyIHNpbmdsZS1lbmQgb3IgcGFpcmVkLWVuZCBSTkEtc2VxIGxpYnJhcmllcyBmb3IgdHJlYXRlZCAoUFMgZGVwbGV0ZWQpIHNhbXBsZXMgYW5kIHVudHJlYXRlZCBzYW1wbGVzLiBUaGVzZSBsaWJyYXJpZXMgd2VyZSBzZXF1ZW5jZWQgdG8gb2J0YWluIGEgY29sbGVjdGlvbiBvZiBSTkEgc2VxdWVuY2luZyByZWFkcyBmb3IgZWFjaCBzYW1wbGUuIFRoZSBlZmZlY3RzIG9mIFBhc2lsbGEgZ2VuZSBkZXBsZXRpb24gb24gc3BsaWNpbmcgZXZlbnRzIGNhbiB0aGVuIGJlIGFuYWx5emVkIGJ5IGNvbXBhcmlzb24gb2YgUk5BIHNlcXVlbmNpbmcgZGF0YSBvZiB0aGUgdHJlYXRlZCAoUFMgZGVwbGV0ZWQpIGFuZCB0aGUgdW50cmVhdGVkIHNhbXBsZXMuClRoZSBnZW5vbWUgb2YgRHJvc29waGlsYSBtZWxhbm9nYXN0ZXIgaXMga25vd24gYW5kIGFzc2VtYmxlZC4gSXQgY2FuIGJlIHVzZWQgYXMgcmVmZXJlbmNlIGdlbm9tZSB0byBlYXNlIHRoaXMgYW5hbHlzaXMuIEluIGEgcmVmZXJlbmNlIGJhc2VkIFJOQS1zZXEgZGF0YSBhbmFseXNpcywgdGhlIHJlYWRzIGFyZSBhbGlnbmVkIChvciBtYXBwZWQpIGFnYWluc3QgYSByZWZlcmVuY2UgZ2Vub21lLCBEcm9zb3BoaWxhIG1lbGFub2dhc3RlciBoZXJlLCB0byBzaWduaWZpY2FudGx5IGltcHJvdmUgdGhlIGFiaWxpdHkgdG8gcmVjb25zdHJ1Y3QgdHJhbnNjcmlwdHMgYW5kIHRoZW4gaWRlbnRpZnkgZGlmZmVyZW5jZXMgb2YgZXhwcmVzc2lvbiBiZXR3ZWVuIHNldmVyYWwgY29uZGl0aW9ucy4KCiMjIERhdGEKClRvIHNhdmUgdGltZSBhbmQgbWVtb3J5LCBwcmUtY291bnRlZCBmaWxlcyBhcmUgYXZhaWxhYmxlIG9ubGluZSAoW2h0dHBzOi8vemVub2RvLm9yZy9yZWNvcmQvMTE4NTEyMiMuV3FHZmJuVzBQQ0ldKGh0dHBzOi8vemVub2RvLm9yZy9yZWNvcmQvMTE4NTEyMiMuV3FHZmJuVzBQQ0kpKQoKLSBHU000NjExNzZfdW50cmVhdF9zaW5nbGUuY291bnRzCi0gR1NNNDYxMTc3X3VudHJlYXRfcGFpcmVkLmNvdW50cwotIEdTTTQ2MTE3OF91bnRyZWF0X3BhaXJlZC5jb3VudHMKLSBHU000NjExNzlfdHJlYXRfc2luZ2xlLmNvdW50cwotIEdTTTQ2MTE4MF90cmVhdF9wYWlyZWQuY291bnRzCi0gR1NNNDYxMTgxX3RyZWF0X3BhaXJlZC5jb3VudHMKLSBHU000NjExODJfdW50cmVhdF9zaW5nbGUuY291bnRzCgojIyBFeGVyY2lzZQoKLSBEb3dubG9hZCB0aGUgZmlsZXMgZnJvbSB0aGUgemVub2RvIGxpbmssIGFuZCB1cGxvYWQgdGhlbSB0byBHYWxheHkKLSBVc2UgREVTZXEyIGFzIHdpdGggdGhlIHByZXZpb3VzIHNlY3Rpb24KLSBGaWx0ZXIgdGhlIHJlc3VsdHMgdXNpbmcgdGhlIGNvbmRpdGlvbiBgYzcgPCAwLjA1IGFuZCAoYzMgPiAxLjAgb3IgYzMgPCAtMS4wKWAKCihPcHRpb25hbCkgU29ydCBieSBhIGNvbHVtbjoKCldlIGNhbiBzb3J0IHRoZSB0YWJsZSBieSB2YWx1ZXMgaW4gYSBwYXJ0aWN1bGFyIGNvbHVtbiAoZWcuIGxvZyBmb2xkIGNoYW5nZSkKCjEuIEZpbHRlciBhbmQgU29ydCAtPiBTb3J0CjIuIFNlbGVjdCB0aGUgbW9zdCByZWNlbnQgZmlsdGVyZWQgdGFibGUgYXMgdGhlIGRhdGFzZXQKMy4gU29ydCBsb2cgZm9sZCBjaGFuZ2UgYnkgRGVzY2VuZGluZyBvcmRlcgoKIyMgUHJlcGFyaW5nIHRoZSBkYXRhIGZvciBHT3JpbGxhCgpXZSB3aWxsIGJlIHVzaW5nIHRoZSBvbmxpbmUgdG9vbCBbR09yaWxsYV0oaHR0cDovL2NibC1nb3JpbGxhLmNzLnRlY2huaW9uLmFjLmlsLykKCjEuIFNlbGVjdCB0aGUgdGFibGUgdGhhdCB5b3UgaGF2ZSBmaWx0ZXJlZCBhbmQgc29ydGVkCjIuIE9wZXJhdGlvbjogS2VlcAozLiBTZWxlY3Qgb25seSBjb2x1bW4gMSAoR2VuZSBJRHMpIGZyb20gTGlzdCBvZiBGaWVsZHMKNC4gU2F2ZSB0aGUgb3V0cHV0dGVkIGRhdGFzZXQgYXMg4oCcREVfR2VuZXMudHh04oCdCgoKIyNNYWtlIGEgYmFja2dyb3VuZCBsaXN0IG9mIGdlbmVzOgoKVGhpcyBpcyB0aGUgbGlzdCBvZiBhbGwgdGhlIGdlbmVzIHlvdSB0ZXN0ZWQgaW4geW91ciBSTkFzZXEgcGlwZWxpbmUuIAoKMS4gRmlsdGVyIGFuZCBTb3J0IC0+IEZpbHRlcgoyLiBTZWxlY3QgdGhlIERFU2VxMiByZXN1bHRzIGZyb20gdGhlIHByZXZpb3VzIHN0ZXAgKnByaW9yIHRvIHAtdmFsdWUgYW5kIGZvbGQtY2hhbmdlIGZpbHRlcmluZyoKMy4gVGV4dCBNYW5pcHVsYXRpb24gLT4gQ3V0CjQuIEtlZXAgdGhlIGZpcnN0IGNvbHVtbgo1LiBEb3dubG9hZCBhbmQgc2F2ZSB0aGUgcmVzdWx0aW5nIGZpbGUgb2YgZ2VuZSBuYW1lcyBhcwrigJxCYWNrZ3JvdW5kLnR4dOKAnQoKIyMgVXNpbmcgR09yaWxsYQoxLiBHbyB0byBodHRwOi8vY2JsLWdvcmlsbGEuY3MudGVjaG5pb24uYWMuaWwvCjIuIFJlYWQgdGhlIOKAnFJ1bm5pbmcgRXhhbXBsZeKAnQoKIVtdKGltYWdlcy9nb3JpbGxhLWV4YW1wbGUucG5nKQoKMy4gQ2hvb3NlIE9yZ2FuaXNtOiBgRHJvc29waGlsYSBtZWxhbm9nYXN0ZXJgCjQuIENob29zZSBydW5uaW5nIG1vZGU6IGBUd28gdW5yYW5rZWQgbGlzdHMgb2YgZ2VuZXNgCjUuIFVwbG9hZCBgREVfR2VuZXMudHh0YCBhcyB0aGUgVGFyZ2V0IHNldC4KNi4gVXBsb2FkIGBCYWNrZ3JvdW5kLnR4dGAgYXMgdGhlIEJhY2tncm91bmQgc2V0Lgo3LiBDaG9vc2UgYW4gT250b2xvZ3k6IGBQcm9jZXNzYAo4LiBgU2VhcmNoIEVucmljaGVkIEdPIHRlcm1zYAoKIyMgVGhlb3J5IFBhcnQgSUk6IFRocmVzaG9sZC1mcmVlCgpGb3IgdGhlc2UgdGVzdHMsIHdlIGRvbid0IGhhdmUgdG8gc3BlY2lmeSBhIHN0YXRpc3RpY2FsIHRocmVzaG9sZCBhbmQgdXNlIHRoZSB0ZXN0IHN0YXRpc3RpY3MgKGxvZyBmb2xkLWNoYW5nZSkgZnJvbSAqYWxsKiBnZW5lcyBhcyB0aGUgaW5wdXQgdG8gdGhlIHRlc3QuIFRoZSBwb3B1bGFyICpHZW5lIFNldCBFbnJpY2htZW50IEFuYWx5c2lzIChHU0VBKSogdXNlcyB0aGlzIGFwcHJvYWNoLiBUaGVzZSB0ZXN0cyBjYW4gYmUgdXNlZCB0byBkZXRlY3QgZGlmZmVyZW50aWFsIGV4cHJlc3Npb24gZm9yIGEgZ3JvdXAgb2YgZ2VuZXMsIGV2ZW4gd2hlbiB0aGUgZWZmZWN0cyBhcmUgdG9vIHNtYWxsIG9yIHRoZXJlIGlzIHRvbyBsaXR0bGUgZGF0YSB0byBkZXRlY3QgdGhlIGdlbmVzIGluZGl2aWR1YWxseS4KCiFbZ3NlYV0oaW1hZ2VzL0dTRUEucG5nKQoKKlN1YnJhbWFuaWFuIGV0IGFsLCBQTkFTIDIwMDUqCgpUaGUgQnJvYWQgaW5zdGl0dXRlIHByb3ZpZGVzIFthIHZlcnNpb24gb2YgR1NFQV0oaHR0cDovL3NvZnR3YXJlLmJyb2FkaW5zdGl0dXRlLm9yZy9nc2VhL2luZGV4LmpzcCkgdGhhdCBjYW4gYmUgcnVuIHZpYSBhIGphdmEgYXBwbGljYXRpb24uIEFub3RoZXIgb3B0aW9uIGlzIFsqKipHZW5lVHJhaWwqKipdKGh0dHA6Ly9nZW5ldHJhaWwuYmlvaW5mLnVuaS1zYi5kZS9lbnJpY2htZW50X2FuYWx5c2lzLnBocD9qcz0xJmNjPTEpIHdoaWNoIGhhcyB0aGUgb3B0aW9uIHRvIGRvIGEgR1NFQS1zdHlsZSBhbmFseXNpcy4KCiMjIFByZXBhcmluZyBkYXRhIGZvciBHZW5lVHJhaWwKCjEuICoqVGV4dCBNYW5pdXB1bGF0aW9uIC0+IEZpbHRlcioqIHdpdGggY29uZGl0aW9uIGBjMyAhPSAnTkEnYAogICArIHNlbGVjdCBgREVTZXEyIHJlc3VsdCBmaWxlLi4uYCBhcyBpbnB1dAoyLiAqKlRleHQgTWFuaXB1bGF0aW9uIC0+IEN1dCoqIAogICArIHNlbGVjdGluZyBjb2x1bW5zIGBjMSBjM2AgYXMgdGhlIGNvbHVtbnMgdG8gY3V0CjMuIERvd25sb2FkIHRoZSBmaWx0ZXJlZCwgY3V0LCB0YWJsZSBhcyBhIGZpbGUgKGUuZy4gYGdzZWEtaW5wdXQudHh0YCkKCiMjIFJ1bm5pbmcgR2VuZVRyYWlsCgpHbyB0byB0aGUgW0dlbmVUcmFpbCB3ZWJzaXRlXShodHRwczovL2dlbmV0cmFpbDIuYmlvaW5mLnVuaS1zYi5kZS8pCgoxLiBVbmRlciBUcmFuc2NyaXB0b21lLCBzZWxlY3QgKipTdGFydCBBbmFseXNpcyoqCjIuIFVuZGVyICoqVXBsb2FkIERhdGEqKiBzZWxlY3QgdGhlIGRhdGEgc291cmNlIHRvIGJlICphIHRleHQgZmlsZSogYW5kIGJyb3dzZSB0byB3aGVyZSB5b3UgZG93bmxvYWRlZCB0aGUgZmlsZSBgZ3NlYS1pbnB1dC50eHRgCjMuIEtlZXAgdGhlIGRlZmF1bHQgY2F0ZWdvcmllcyBhbmQgKipTdGFydCBBbmFseXNpcyoqCjQuIEV4cGxvcmUgdGhlIHJlc3VsdHMgdG8gc2VlIHdoYXQgcGF0aHdheXMgYXJlIHJlcG9ydGVkCgo=