This page contains some supplementary data for our paper Reconsidering the significance of genomic word frequencies (written with L. Noé and G. Kucherov). A preliminary version of the manuscript can be found in arXiv. The final version will appear in the journal Trends in Genetics towards the end of 2007.

spectrum example A main finding is that genomic oligonucleotide frequencies follow a Pareto-lognormal distribution in a wide variety of organisms, including prokaryotes and eukaryotes. Why? We speculate that such features are fundamentally attributable to copying processes: genomic duplications, retrotranscription, tandem repeats and so on. In other words, global features of genomic spectra arise from duplicative evolutionary processes, and not necessarily from intricate word-level selection on point mutations and deletions that are enacting adaptation and conservation, or simply obeying structural constraints. In practice, the heavy tail of word frequency distributions means that caution should be exercised when inferring functionality of motifs from frequency alone, especially if overrepresentation is related to word occurrences in random texts. These random text models include Markov and Bernoulli models, which are routinely employed in bioinformatics practice to assess over- and underrepresentation. Such assessment is often needed in various contexts, including discovery of sequence motifs (e.g., transcription factor binding sites) and mobile elements, homology search (masking repeats and low-complexity regions) and genome assembly (repeat-masking).

Examples of genomic word frequency distributions

Spectra are computed and fitted to a double Pareto-lognormal distribution for cases when the spectrum spans at least a few hundred frequencies. (By spectrum, we mean the function W(n), where W(n) is the number of words that occur n times in a given genomic sequence.)

Bacteria

Archaea

Eukaryota

Random sequence generators

Random sequences were generated using my Java executable programs from RandomSequences.jar. The programs are the following.

ca.umontreal.iro.genome.ShuffleDNASequence file window_size
Produces a shuffling of the input Fasta sequence in file. The sequence is partitioned into contiguous pieces with window_size non-ambiguous (A, C, G, T) nucleotides. Non-ambiguous nucleotides are garbled within each window by picking a uniform random permutation.
ca.umontreal.iro.genome.hash.RandomSequenceSameKmer file k
Produces a random sequence with the same k-mer distribution and length as the sequence in the Fasta file. In other words, produces a Markov model of order k-1 (if k>1) or a Bernoulli model (if k=1).
ca.umontreal.iro.genome.hash.RandomSequenceCopyPaste [-initlen n] [-gc fraction] [-copy m] [-finallen N] [-events K]
Produces a random sequence using the copy-and-paste model outlined in the paper. Initial length and GC content are given by the initlen and gc switches (GC fraction should bet between 0 and 1). The initial sequence grows by copying random pieces of length m. Copying stops when the sequence length surpasses N of finallen or after K copy events, whichever happens earlier.

You can launch the programs by using the Java engine you have: java -cp RandomSequences.jar ca.umontreal.iro.genome.... It may be necessary to allow for larger memory usage: java -Xmx2000M -cp ...

I very much appreciate if you send me an e-mail in case you decide using my programs.

CC-GNU GPL
This software is licensed under the CC-GNU GPL.