{"id":3204,"date":"2017-04-01T18:28:42","date_gmt":"2017-04-01T12:58:42","guid":{"rendered":"https:\/\/www.wikitechy.com\/technology\/?p=3204"},"modified":"2017-04-01T18:28:42","modified_gmt":"2017-04-01T12:58:42","slug":"faster-process-sorted-array-unsorted-array","status":"publish","type":"post","link":"https:\/\/www.wikitechy.com\/technology\/faster-process-sorted-array-unsorted-array\/","title":{"rendered":"[ Solved -11 Answers] JAVA &#8211; Why is it faster to process a sorted array than an unsorted array"},"content":{"rendered":"<p><label class=\"label label-warning\">PROBLEM:<\/label><\/p>\n<p>A piece of C++ code that seems very peculiar. For some reason, sorting the data miraculously makes the code almost six times faster.<\/p>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">c++ code<\/span> <\/div> <pre class=\"language-cpp code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-cpp code-embed-code\">#include &lt;algorithm&gt;<br\/>#include &lt;ctime&gt;<br\/>#include &lt;iostream&gt;<br\/><br\/>int main()<br\/>{<br\/>    \/\/ Generate data<br\/>    const unsigned arraySize = 32768;<br\/>    int data[arraySize];<br\/><br\/>    for (unsigned c = 0; c &lt; arraySize; ++c)<br\/>        data[c] = std::rand() % 256;<br\/><br\/>    \/\/ !!! With this, the next loop runs faster<br\/>    std::sort(data, data + arraySize);<br\/>\/\/ Test<br\/>    clock_t start = clock();<br\/>    long long sum = 0;<br\/> for (unsigned i = 0; i &lt; 100000; ++i)<br\/>    {<br\/>        \/\/ Primary loop<br\/>        for (unsigned c = 0; c &lt; arraySize; ++c)<br\/>        {<br\/>            if (data[c] &gt;= 128)<br\/>                sum += data[c];<br\/>        }<br\/>    }<br\/> double elapsedTime = static_cast&lt;double&gt;(clock() - start) \/ CLOCKS_PER_SEC;<br\/> std::cout &lt;&lt; elapsedTime &lt;&lt; std::endl;<br\/> std::cout &lt;&lt; &quot;sum = &quot; &lt;&lt; sum &lt;&lt; std::endl;<br\/>}<\/code><\/pre> <\/div>\n<ul>\n<li>Without\u00a0std::sort(data, data + arraySize);, the code runs in 11.54 seconds.<\/li>\n<li>With the sorted data, the code runs in 1.93 seconds.<\/li>\n<\/ul>\n<p><strong>Initially, this might be just a language or compiler anomaly. So a Trail in Java.<\/strong><\/p>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">import java.util.Arrays;<br\/>import java.util.Random;<br\/><br\/>public class Main<br\/>{<br\/>    public static void main(String[] args)<br\/>    {<br\/>        \/\/ Generate data<br\/>        int arraySize = 32768;<br\/>        int data[] = new int[arraySize];<br\/><br\/>        Random rnd = new Random(0);<br\/>        for (int c = 0; c &lt; arraySize; ++c)<br\/>            data[c] = rnd.nextInt() % 256;<br\/><br\/>        \/\/ !!! With this, the next loop runs faster<br\/>        Arrays.sort(data);<br\/>\/\/ Test<br\/>        long start = System.nanoTime();<br\/>        long sum = 0;<br\/><br\/>        for (int i = 0; i &lt; 100000; ++i)<br\/>        {<br\/>            \/\/ Primary loop<br\/>            for (int c = 0; c &lt; arraySize; ++c)<br\/>            {<br\/>                if (data[c] &gt;= 128)<br\/>                    sum += data[c];<br\/>            }<br\/>        }<br\/><br\/>        System.out.println((System.nanoTime() - start) \/ 1000000000.0);<br\/>        System.out.println(&quot;sum = &quot; + sum);<br\/>    }<br\/>}<\/code><\/pre> <\/div>\n<p>With a somewhat similar but less extreme result.<\/p>\n<p><label class=\"label label-info\">SOLUTION 1:<\/label><\/p>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">c ++ code<\/span> <\/div> <pre class=\"language-cpp code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-cpp code-embed-code\">if (data[c] &gt;= 128) sum += data[c];<\/code><\/pre> <\/div>\n<ul>\n<li>Notice that the data is evenly distributed between 0 and 255.<\/li>\n<li>When the data is sorted, roughly the first half of the iterations will not enter the if-statement.<\/li>\n<li>After that, they will all enter the if-statement.<\/li>\n<li>This is very friendly to the branch predictor since the branch consecutively goes the same direction many times.<\/li>\n<li>Even a simple saturating counter will correctly predict the branch except for the few iterations after it switches direction.<\/li>\n<\/ul>\n<h4 id=\"quick-visualization\"><span style=\"color: #ff6600;\"><b>Quick visualization:<\/b><\/span><\/h4>\n<p>T = branch taken N = branch not taken data[] = 0, 1, 2, 3, 4, &#8230; 126, 127, 128, 129, 130, &#8230; 250, 251, 252, &#8230; branch = N N N N N &#8230; N N T T T &#8230; T T T &#8230; = NNNNNNNNNNNN &#8230; NNNNNNNTTTTTTTTT &#8230; TTTTTTTTTT (easy to predict)<\/p>\n<ul>\n<li>However, when the data is completely random, the branch predictor is rendered useless because it can&#8217;t predict random data.<\/li>\n<\/ul>\n<p>data[] = 226, 185, 125, 158, 198, 144, 217, 79, 202, 118, 14, 150, 177, 182, 133, &#8230; branch = T, T, N, T, T, T, T, N, T, N, N, T, T, T, N &#8230; = TTNTTTTNTNNTTTN &#8230; (completely random &#8211; hard to predict)<\/p>\n<p>If the compiler isn&#8217;t able to optimize the branch into a conditional move,<\/p>\n<h4 id=\"replace\"><span style=\"color: #808000;\"><strong>Replace:<\/strong><\/span><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">c++ code<\/span> <\/div> <pre class=\"language-cpp code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-cpp code-embed-code\">if (data[c] &gt;= 128) <br\/>sum += data[c]; <\/code><\/pre> <\/div>\n<h4 id=\"with\"><strong><span style=\"color: #800080;\">with:<\/span><\/strong><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">c++ code<\/span> <\/div> <pre class=\"language-cpp code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-cpp code-embed-code\">int t = (data[c] - 128) &gt;&gt; 31;<br\/>sum += ~t &amp; data[c]; <\/code><\/pre> <\/div>\n<ul>\n<li>This eliminates the branch and replaces it with some bitwise operations.<\/li>\n<\/ul>\n<h4 id=\"c-visual-studio-2010-x64-release\"><strong><span style=\"color: #ff6600;\">C++ &#8211; Visual Studio 2010 &#8211; x64 Release<\/span><\/strong><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">c ++ code<\/span> <\/div> <pre class=\"language-cpp code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-cpp code-embed-code\">\/\/ Branch - Random <br\/>seconds = 11.777 <br\/>\/\/ Branch - Sorted <br\/>seconds = 2.352 <br\/>\/\/ Branchless - Random <br\/>seconds = 2.564 <br\/>\/\/ Branchless \u2013 Sorted<br\/>seconds = 2.587 <\/code><\/pre> <\/div>\n<h4 id=\"java-netbeans-7-1-1-jdk-7-x64\"><strong><span style=\"color: #808000;\">Java &#8211; Netbeans 7.1.1 JDK 7 &#8211; x64<\/span><\/strong><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">\/\/ Branch - Random <br\/>seconds = 10.93293813 <br\/>\/\/ Branch - Sorted <br\/>seconds = 5.643797077 <br\/>\/\/ Branchless \u2013 Random<br\/> seconds = 3.113581453 <br\/>\/\/ Branchless \u2013 Sorted<br\/> seconds = 3.186068823 <\/code><\/pre> <\/div>\n<ul>\n<li><b>With the Branch:<\/b>There is a huge difference between the sorted and unsorted data.<\/li>\n<li><b>With the Hack:<\/b>There is no difference between sorted and unsorted data.<\/li>\n<li>In the C++ case, the hack is actually a tad slower than with the branch when the data is sorted.<\/li>\n<\/ul>\n<p><label class=\"label label-info\">SOLUTION 2:<\/label><\/p>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">c++ code<\/span> <\/div> <pre class=\"language-cpp code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-cpp code-embed-code\">if (data[c] &gt;= 128)<br\/> sum += data[c]; <\/code><\/pre> <\/div>\n<ul>\n<li>The meaning of this particular\u00a0if&#8230; else&#8230;\u00a0branch is to add something when a condition is satisfied.<\/li>\n<li>This type of branch can be easily transformed into a\u00a0<b>conditional move<\/b>statement, which would be compiled into a conditional move instruction:\u00a0cmovl, in an\u00a0x86\u00a0system.<\/li>\n<li>The branch and thus the potential branch prediction penalty is removed.<\/li>\n<\/ul>\n<ul>\n<li>In\u00a0C, thus\u00a0C++, the statement, which would compile directly (without any optimization) into the conditional move instruction in\u00a0x86, is the ternary operator\u00a0&#8230; ? &#8230; : &#8230;.<\/li>\n<li>So it can be rewritten as:<\/li>\n<\/ul>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">c++ code<\/span> <\/div> <pre class=\"language-cpp code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-cpp code-embed-code\">sum += data[c] &gt;=128 ? data[c] : 0; <\/code><\/pre> <\/div>\n[ad type=&#8221;banner&#8221;]\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">x86<br\/>\/\/ Branch - Random <br\/>seconds = 8.885 <br\/>\/\/ Branch - Sorted <br\/>seconds = 1.528<br\/> \/\/ Branchless - Random seconds = 3.716 <br\/>\/\/ Branchless - Sorted seconds = 3.71 <\/code><\/pre> <\/div>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">x64<br\/>\/\/ Branch \u2013 Random<br\/> seconds = 11.302<br\/> \/\/ Branch \u2013 Sorted<br\/> seconds = 1.830 <br\/>\/\/ Branchless - Random <br\/>seconds = 2.736<br\/> \/\/ Branchless \u2013 Sorted<br\/> seconds = 2.737 <\/code><\/pre> <\/div>\n[ad type=&#8221;banner&#8221;]\n<p><label class=\"label label-info\">SOLUTION 3:<\/label><\/p>\n<p>Thus from the previous fix, lets investigate the\u00a0x86\u00a0assembly they generate. For simplicity, we use two functions\u00a0max1\u00a0and\u00a0max2.<\/p>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">max1\u00a0uses the conditional branch\u00a0if... else ...:<br\/>int max1(int a, int b) <br\/>{ <br\/>if (a &gt; b)<br\/> return a;<br\/> else return b; <br\/>} <\/code><\/pre> <\/div>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">max2\u00a0uses the ternary operator\u00a0... ? ... : ...:<br\/>int max2(int a, int b)<br\/> { <br\/>return a &gt; b ? a : b; <br\/>} <\/code><\/pre> <\/div>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">C++ code<\/span> <\/div> <pre class=\"language-cpp code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-cpp code-embed-code\">max1 <br\/>movl %edi, -4(%rbp) <br\/>movl %esi, -8(%rbp) <br\/>movl -4(%rbp), %eax <br\/>cmpl -8(%rbp), %eax <br\/>jle .L2 <br\/>movl -4(%rbp), %eax <br\/>movl %eax, -12(%rbp) <br\/>jmp .L4 <br\/>.L2: <br\/>movl -8(%rbp), %eax <br\/>movl %eax, -12(%rbp) <br\/>.L4: <br\/>movl -12(%rbp), %eax <br\/>leave <br\/>ret <br\/>:max2 <br\/>movl %edi, -4(%rbp) <br\/>movl %esi, -8(%rbp) <br\/>movl -4(%rbp), %eax <br\/>cmpl %eax, -8(%rbp) <br\/>cmovge -8(%rbp), %eax <br\/>leave <br\/>ret <\/code><\/pre> <\/div>\n<ul>\n<li>max2\u00a0uses much less code due to the usage of instruction\u00a0cmovge<\/li>\n<li>But the real gain is that\u00a0max2does not involve branch jumps,\u00a0jmp, which would have a significant performance penalty if the predicted result is not right<\/li>\n<li>In a typical\u00a0x86\u00a0processor, the execution of an instruction is divided into several stages.<\/li>\n<li>Thus they have different hardware to deal with different stages.<\/li>\n<li>So we do not have to wait for one instruction to finish to start a new one.<\/li>\n<li>This is called\u00a0<b>pipelining<\/b>.<\/li>\n<\/ul>\n<p><label class=\"label label-info\">SOLUTION 4:<\/label><\/p>\n<h4 id=\"starting-with-the-original-loop\">\u00a0<span style=\"color: #ff6600;\">Starting with the original loop:<\/span><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">for (unsigned i = 0; i &lt; 100000; ++i)<br\/> { <br\/>for (unsigned j = 0; j &lt; arraySize; ++j)<br\/> { <br\/>if (data[j] &gt;= 128)<br\/> sum += data[j]; <br\/>} <br\/>} <\/code><\/pre> <\/div>\n<h4 id=\"with-loop-interchange-we-can-change-this-loop-to\"><span style=\"color: #808000;\">With loop interchange, we can change this loop to:<\/span><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">for (unsigned j = 0; j &lt; arraySize; ++j) <br\/>{ <br\/>for (unsigned i = 0; i &lt; 100000; ++i)<br\/> { <br\/>if (data[j] &gt;= 128)<br\/> sum += data[j]; <br\/>} <br\/>} <\/code><\/pre> <\/div>\n<p>if&#8221; conditional is constant throughout the execution of the &#8220;i&#8221; loop, so you can hoist the &#8220;if&#8221; out:<\/p>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">for (unsigned j = 0; j &lt; arraySize; ++j) <br\/>{ <br\/>if (data[j] &gt;= 128) <br\/>{ <br\/>for (unsigned i = 0; i &lt; 100000; ++i) <br\/>{ <br\/>sum += data[j]; <br\/>} } } <\/code><\/pre> <\/div>\n<p>the inner loop can be collapsed into one single expression, assuming the floating point model allows it<\/p>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">for (unsigned j = 0; j &lt; arraySize; ++j)<br\/> { <br\/>if (data[j] &gt;= 128)<br\/> { <br\/>sum += data[j] * 100000;<br\/> } <br\/>} <\/code><\/pre> <\/div>\n<p><label class=\"label label-info\">SOLUTION 5:<\/label><\/p>\n<ul>\n<li>The Valgrind tool cachegrind has a branch-predictor simulator, enabled by using the &#8211;branch-sim=yes flag.<\/li>\n<li>Running it over the examples in this question, with the number of outer loops reduced to 10000 and compiled with g++, gives these results:<\/li>\n<\/ul>\n<h4 id=\"sorted\"><span style=\"color: #800080;\"><strong>Sorted:<\/strong><\/span><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">==32551== Branches:        656,645,130  (  656,609,208 cond +    35,922 ind)<br\/>==32551== Mispredicts:         169,556  (      169,095 cond +       461 ind)<br\/>==32551== Mispred rate:            0.0% (          0.0%     +       1.2%   )<\/code><\/pre> <\/div>\n<h4 id=\"unsorted\"><strong><span style=\"color: #ff6600;\">Unsorted:<\/span><\/strong><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">==32555== Branches:        655,996,082  (  655,960,160 cond +  35,922 ind)<br\/>==32555== Mispredicts:     164,073,152  (  164,072,692 cond +     460 ind)<br\/>==32555== Mispred rate:           25.0% (         25.0%     +     1.2%   )<\/code><\/pre> <\/div>\n<ul>\n<li>Drilling down into the line-by-line output produced by cg_annotate we see for the loop in question:<\/li>\n<\/ul>\n<h4 id=\"sorted-2\"><span style=\"color: #808000;\"><strong>Sorted:<\/strong><\/span><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">Bc    Bcm Bi Bim<br\/>      10,001      4  0   0      for (unsigned i = 0; i &lt; 10000; ++i)<br\/>           .      .  .   .     \t\t\t {<br\/>           .      .  .   .        \t\t\t  \/\/ primary loop<br\/> 327,690,000 10,016  0   0          for (unsigned c = 0; c &lt; \t\t\t\tarraySize; ++c)<br\/>           .      .  .   .        \t\t\t  {<br\/> 327,680,000 10,006  0   0              if (data[c] &gt;= 128)<br\/>           0      0  0   0                  sum += data[c];<br\/>           .      .  .   .         \t\t }<br\/>           .      .  .   .     \t\t\t }<\/code><\/pre> <\/div>\n<h4 id=\"unsorted-2\"><span style=\"color: #800080;\"><strong>Unsorted:<\/strong><\/span><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\"><br\/>          Bc         Bcm Bi Bim<br\/>      10,001           4  0   0      for (unsigned i = 0; i &lt; 10000; ++i)<br\/>           .           .  .   .     \t\t {<br\/>           .           .  .   .         \t\t \/\/ primary loop<br\/> 327,690,000      10,038  0   0          for (unsigned c = 0; c &lt; \t\t\t\t\tarraySize; ++c)<br\/>           .           .  .   .        \t\t  {<br\/> 327,680,000 164,050,007  0   0              if (data[c] &gt;= 128)<br\/>           0           0  0   0                  \tsum += data[c];<br\/>           .           .  .   .         \t\t }<br\/>           .           .  .   .     \t\t }<\/code><\/pre> <\/div>\n[ad type=&#8221;banner&#8221;]\n<ul>\n<li>the problematic line &#8211; in the unsorted version the if (data[c] &gt;= 128) line is causing 164,050,007 mispredicted conditional branches (Bcm) under cachegrind&#8217;s branch-predictor model, whereas it&#8217;s only causing 10,006 in the sorted version.<\/li>\n<\/ul>\n<p><label class=\"label label-info\">SOLUTION 6:<\/label><\/p>\n<ul>\n<li>On Linux you can use the performance counters subsystem to accomplish The Valgrind tool cachegrind has a branch-predictor simulator, enabled by using the &#8211;branch-sim=yes flag, but with native performance using CPU counters.<\/li>\n<\/ul>\n<h4 id=\"sorted-3\"><strong><span style=\"color: #ff6600;\">Sorted:<\/span><\/strong><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">Performance counter stats for &#039;.\/sumtest_sorted&#039;: <br\/>11808.095776 task-clock # 0.998 CPUs utilized <br\/>1,062 context-switches # 0.090 K\/sec <br\/>14 CPU-migrations # 0.001 K\/sec <br\/>337 page-faults # 0.029 K\/sec <br\/>26,487,882,764 cycles # 2.243 GHz <br\/>41,025,654,322 instructions # 1.55 insns per cycle 6,558,871,379 branches # 555.455 M\/sec <br\/>567,204 branch-misses # 0.01% of all branches 11.827228330 seconds time elapsed <\/code><\/pre> <\/div>\n<h4 id=\"unsorted-3\"><span style=\"color: #808000;\"><b>Unsorted:<\/b><\/span><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">Performance counter stats for &#039;.\/sumtest_unsorted&#039;: 28877.954344 task-clock # 0.998 CPUs utilized<br\/> 2,584 context-switches # 0.089 K\/sec<br\/> 18 CPU-migrations # 0.001 K\/sec <br\/>335 page-faults # 0.012 K\/sec <br\/>65,076,127,595 cycles # 2.253 GHz <br\/>41,032,528,741 instructions # 0.63 insns per cycle 6,560,579,013 branches # 227.183 M\/sec<br\/> 1,646,394,749 branch-misses # 25.10% of all branches 28.935500947 seconds time elapsed <\/code><\/pre> <\/div>\n<ul>\n<li>It can also do source code annotation with dissassembly.<\/li>\n<li>perf record -e branch-misses .\/sumtest_unsorted<\/li>\n<li>perf annotate -d sumtest_unsorted<\/li>\n<\/ul>\n<p>Percent | \u00a0 Source code &amp; Disassembly of sumtest_unsorted<\/p>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">...<br\/>         :                      sum += data[c];<br\/>    0.00 :        400a1a:       mov    -0x14(%rbp),%eax<br\/>   39.97 :        400a1d:       mov    %eax,%eax<br\/>    5.31 :        400a1f:       mov    -0x20040(%rbp,%rax,4),%eax<br\/>    4.60 :        400a26:       cltq   <br\/>    0.00 :        400a28:       add    %rax,-0x30(%rbp)<br\/>...<\/code><\/pre> <\/div>\n<p><label class=\"label label-info\">SOLUTION 7:<\/label><\/p>\n<p>A common way to eliminate branch prediction is that, to work in managed languages -a table lookup instead of using a branch<\/p>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">\/\/ generate data<br\/>int arraySize = 32768;<br\/>int[] data = new int[arraySize];<br\/><br\/>Random rnd = new Random(0);<br\/>for (int c = 0; c &lt; arraySize; ++c)<br\/>    data[c] = rnd.Next(256);<br\/><br\/>\/\/ Too keep the spirit of the code in-tact we will make a separate lookup table<br\/>\/\/ (Assume we cannot modify &#039;data&#039; or the number of loops)<br\/>int[] lookup = new int[256];<br\/><br\/>for (int c = 0; c &lt; 256; ++c)<br\/>    lookup[c] = (c &gt;= 128) ? c : 0;<br\/>\/\/ test<br\/>DateTime startTime = System.DateTime.Now;<br\/>long sum = 0;<br\/><br\/>for (int i = 0; i &lt; 100000; ++i)<br\/>{<br\/>    \/\/ primary loop<br\/>    for (int j = 0; j &lt; arraySize; ++j)<br\/>    {<br\/>        \/\/ here you basically want to use simple operations - so no <br\/>        \/\/ random branches, but things like &amp;, |, *, -, +, etc are fine.<br\/>        sum += lookup[data[j]];<br\/>    }<br\/>}<br\/><br\/>DateTime endTime = System.DateTime.Now;<br\/>Console.WriteLine(endTime - startTime);<br\/>Console.WriteLine(&quot;sum = &quot; + sum);<br\/><br\/>Console.ReadLine();<\/code><\/pre> <\/div>\n<p><label class=\"label label-info\">SOLUTION 8:<\/label><\/p>\n<ul>\n<li>measure the performance of this loop with different conditions:<\/li>\n<\/ul>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">for (int i = 0; i &lt; max; i++) if (condition) sum++; <\/code><\/pre> <\/div>\n<p>Here are the timings of the loop with different True-False patterns:<\/p>\n<blockquote><p>Condition \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0Pattern\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 Time (ms)<\/p>\n<p>(i &amp; 0\u00d780000000) == 0\u00a0 \u00a0 \u00a0T repeated\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 322<\/p>\n<p>(i &amp; 0xffffffff) == 0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0F repeated\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 276<\/p>\n<p>(i &amp; 1) == 0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 TF alternating\u00a0\u00a0\u00a0 \u00a0 760<\/p>\n<p>(i &amp; 3) == 0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 TFFFTFFF\u2026 \u00a0 \u00a0 \u00a0 513<\/p>\n<p>(i &amp; 2) == 0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 TTFFTTFF\u2026 \u00a0 \u00a0 \u00a01675<\/p>\n<p>(i &amp; 4) == 0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 TTTTFFFFTTTTFFFF1275<\/p>\n<p>(i &amp; 8) == 0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a08T 8F 8T 8F \u2026\u00a0\u00a0\u00a0\u00a0 \u00a0 752<\/p>\n<p>(i &amp; 16) == 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 16T 16F 16T 16F \u2026 \u00a0 \u00a0490<\/p><\/blockquote>\n<ul>\n<li>A \u201c<b>bad<\/b>\u201d true-false pattern can make an if-statement up to six times slower than a \u201c<b>good<\/b>\u201d pattern!<\/li>\n<li>\u00a0Thus which pattern is good and which is bad depends on the exact instructions generated by the compiler and on the specific processor.<\/li>\n<\/ul>\n<p><label class=\"label label-info\">SOLUTION 9:<\/label><\/p>\n<ul>\n<li>The\u00a0way to avoid branch prediction errors is to build a lookup table, and index it using the data<\/li>\n<li>By using the 0\/1 value of the decision bit as an index into an array, we can make code that will be equally fast whether the data is sorted or not sorted.<\/li>\n<li>Our code will always add a value, but when the decision bit is 0, we will add the value somewhere we don&#8217;t care<\/li>\n<\/ul>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">\/\/ Test<br\/>clock_t start = clock();<br\/>long long a[] = {0, 0};<br\/>long long sum;<br\/><br\/>for (unsigned i = 0; i &lt; 100000; ++i)<br\/>{<br\/>    \/\/ Primary loop<br\/>    for (unsigned c = 0; c &lt; arraySize; ++c)<br\/>    {<br\/>        int j = (data[c] &gt;&gt; 7);<br\/>        a[j] += data[c];<br\/>    }<br\/>}<br\/><br\/>double elapsedTime = static_cast&lt;double&gt;(clock() - start) \/ CLOCKS_PER_SEC;<br\/>sum = a[1];<\/code><\/pre> <\/div>\n<p><strong>This code wastes half of the adds, but never has a branch prediction failure.<\/strong><\/p>\n<ul>\n<li>The technique of indexing into an array, instead of using an if statement, can be used for deciding which pointer to use.<\/li>\n<li>A library that implemented binary trees, and instead of having two named pointers (pLeft and pRight or whatever) had a length-2 array of pointers, and used the &#8220;decision bit&#8221; technique to decide which one to follow.<\/li>\n<li>For example, instead of:<\/li>\n<\/ul>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">if (x &lt; node-&gt;value)<br\/>    node = node-&gt;pLeft;<br\/>else<br\/>    node = node-&gt;pRight;<br\/>this library would do something like:<br\/><br\/>i = (x &lt; node-&gt;value);<br\/>node = node-&gt;link[i];<\/code><\/pre> <\/div>\n[ad type=&#8221;banner&#8221;]\n<p><label class=\"label label-info\">SOLUTION 10:<\/label><\/p>\n<ul>\n<li>In the sorted case, you can do better than relying on successful branch prediction or any branchless comparison trick: completely remove the branch.<\/li>\n<li>The array is partitioned in a contiguous zone with data &lt; 128 and another with data &gt;= 128. So you should find the partition point with a dichotomic search (using Lg(arraySize) = 15 comparisons), then do a straight accumulation from that point.<\/li>\n<\/ul>\n<h4 id=\"something-like-unchecked\"><span style=\"color: #800080;\"><b>Something like (unchecked)<\/b><\/span><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">int i= 0, j, k= arraySize;<br\/>while (i &lt; k)<br\/>{<br\/>  j= (i + k) &gt;&gt; 1;<br\/>  if (data[j] &gt;= 128)<br\/>    k= j;<br\/>  else<br\/>    i= j;<br\/>}<br\/>sum= 0;<br\/>for (; i &lt; arraySize; i++)<br\/>  sum+= data[i];<\/code><\/pre> <\/div>\n<h4 id=\"slightly-more-obfuscated\"><span style=\"color: #ff6600;\"><b>slightly more obfuscated<\/b><\/span><\/h4>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">int i, k, j= (i + k) &gt;&gt; 1;<br\/>for (i= 0, k= arraySize; i &lt; k; (data[j] &gt;= 128 ? k : i)= j)<br\/>  j= (i + k) &gt;&gt; 1;<br\/>for (sum= 0; i &lt; arraySize; i++)<br\/>  sum+= data[i];<\/code><\/pre> <\/div>\n<p><label class=\"label label-info\">SOLUTION 11:<\/label><\/p>\n<p>The diagram shows\u00a0why the branch predictor gets confused.<\/p>\n<p><img fetchpriority=\"high\" decoding=\"async\" class=\"aligncenter wp-image-3228 size-full\" src=\"https:\/\/www.wikitechy.com\/technology\/wp-content\/uploads\/2017\/03\/Pinj.png\" alt=\"\" width=\"604\" height=\"124\" \/><\/p>\n<ul>\n<li>Each element in the original code is a random value<\/li>\n<\/ul>\n<div class=\"code-embed-wrapper\"> <div class=\"code-embed-infos\"> <span class=\"code-embed-name\">java code<\/span> <\/div> <pre class=\"language-java code-embed-pre line-numbers\"  data-start=\"1\" data-line-offset=\"0\"><code class=\"language-java code-embed-code\">data[c] = std::rand() % 256;<\/code><\/pre> <\/div>\n<ul>\n<li>so the predictor will change sides as the std::rand() blow.<\/li>\n<li>Once it&#8217;s sorted, the predictor will first move into a state of strongly not taken and when the values change to the high value the predictor will in three runs through change all the way from strongly not taken to strongly taken.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Why is it faster to process a sorted array than an unsorted array -When the data is sorted, roughly the first half of the<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2139],"tags":[6042,6043,6040,6038,6041,6047,6034,6030,6046,6044,6045,6036,6033,6029,6015,6035,6031,6032,6037,6039],"class_list":["post-3204","post","type-post","status-publish","format-standard","hentry","category-java","tag-branch-prediction-program-in-java","tag-c-branch-prediction","tag-difference-between-sorted-and-unsorted-array","tag-does-a-sorted-array-run-faster-than-an-unsorted-array-in-java","tag-explain-how-you-know-whether-google-keeps-its-inventory-of-webpage-content-sorted","tag-how-does-branch-prediction-work","tag-is-in-sorted-array-not-faster-than-unsorted-array","tag-is-faster-than","tag-java-branch-prediction","tag-sorted-and-unsorted-array-in-c","tag-sorted-array-c","tag-why-is-ab-0-faster-than-a-0-b-0-in-java","tag-why-is-faster-than-list","tag-why-is-one-loop-so-much-slower-than-two-loops","tag-why-is-printing-b-dramatically-slower-than-printing","tag-why-is-processing-a-sorted-array-slower-than-an-unsorted-array-javas-arraylist-indexof","tag-why-is-processing-a-sorted-array-not-faster-than-an-unsorted-array-in-python","tag-why-is-processing-a-sorted-array-slower-than-an-unsorted-array","tag-why-is-this-c-code-faster-than-my-hand-written-assembly-for-testing-the-collatz-conjecture","tag-why-processing-a-sorted-array-might-be-faster-than-unsorted"],"_links":{"self":[{"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/posts\/3204","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/comments?post=3204"}],"version-history":[{"count":0,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/posts\/3204\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/media?parent=3204"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/categories?post=3204"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/tags?post=3204"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}