Alexander Lobakin d206121faf
skbuff: micro-optimize {,__}skb_header_pointer() ...
{,__}skb_header_pointer() helpers exist mainly for preventing
accesses-beyond-end of the linear data.
In the vast majorify of cases, they bail out on the first condition.
All code going after is mostly a fallback.
Mark the most common branch as 'likely' one to move it in-line.
Also, skb_copy_bits() can return negative values only when the input
arguments are invalid, e.g. offset is greater than skb->len. It can
be safely marked as 'unlikely' branch, assuming that hotpath code
provides sane input to not fail here.
These two bump the throughput with a single Flow Dissector pass on
every packet (e.g. with RPS or driver that uses eth_get_headlen())
on 20 Mbps per flow/core.
Signed-off-by: Alexander Lobakin <alobakin@pm.me >
Signed-off-by: David S. Miller <davem@davemloft.net >
2021-03-14 14:48:02 -07:00
2021-03-01 11:45:04 +01:00
2021-02-16 12:09:52 +01:00
2021-02-22 09:45:23 -08:00
2021-02-09 12:26:40 +01:00
2021-02-16 14:47:46 -08:00
2021-02-24 10:25:37 -08:00
2021-03-08 11:59:17 +01:00
2021-02-15 21:28:53 +01:00
2021-02-24 10:35:29 -08:00
2021-03-12 15:29:34 -08:00
2021-02-08 12:53:34 +01:00
2021-02-05 15:24:59 +02:00
2021-02-23 15:05:10 -08:00
2021-02-12 14:00:07 +00:00
2021-03-04 15:45:03 -07:00
2021-02-21 14:44:44 -08:00
2021-03-02 15:33:00 +01:00
2021-02-22 09:27:18 -08:00
2021-02-08 13:23:42 +02:00
2021-02-22 13:29:55 -08:00
2021-02-25 10:17:31 -08:00
2021-03-08 11:59:17 +01:00
2021-03-08 15:16:30 -08:00
2021-02-15 18:28:30 +01:00
2021-02-26 15:49:51 -07:00
2021-02-26 09:41:04 -08:00
2021-02-08 08:33:16 -07:00
2021-02-10 16:38:02 +01:00
2021-03-01 10:15:07 -07:00
2021-02-23 09:54:51 -07:00
2021-02-26 11:51:48 -08:00
2021-02-26 11:51:47 -08:00
2021-02-26 12:28:03 -08:00
2021-02-26 13:23:52 -08:00
2021-03-10 01:06:34 +01:00
2021-02-16 15:23:23 -08:00
2021-02-26 09:41:03 -08:00
2021-02-17 14:07:48 +01:00
2021-02-14 12:37:48 -08:00
2021-02-22 09:45:23 -08:00
2021-02-14 18:15:15 +00:00
2021-02-16 12:01:32 +09:00
2021-02-16 12:01:32 +09:00
2021-02-11 18:31:48 +01:00
2021-02-28 12:06:45 -08:00
2021-02-26 09:41:03 -08:00
2021-02-11 09:45:28 -05:00
2021-02-26 13:59:32 -08:00
2021-02-24 21:05:28 +05:30
2021-02-24 21:04:54 +05:30
2021-02-24 09:54:24 -08:00
2021-02-26 13:59:32 -08:00
2021-02-17 14:12:42 +01:00
2021-02-17 14:12:43 +01:00
2021-03-14 14:48:02 -07:00
2021-02-16 09:59:41 +01:00
2021-02-08 12:28:07 +01:00
2021-03-10 01:06:34 +01:00
2021-02-26 09:41:04 -08:00
2021-02-24 13:38:28 -08:00
2021-02-07 14:51:11 -08:00
2021-02-09 12:43:42 -05:00
2021-02-24 10:13:55 -08:00
2021-02-26 09:40:59 -08:00
2021-02-24 13:38:32 -08:00
2021-02-11 19:56:15 +01:00
2021-02-24 13:38:32 -08:00
2021-02-24 13:38:35 -08:00
2021-02-15 10:49:11 +00:00
2021-02-23 11:29:52 -08:00
2021-02-11 13:24:44 -08:00
2021-02-23 13:39:45 -08:00
2021-02-16 13:14:06 -08:00
2021-02-26 09:41:05 -08:00
2021-02-18 23:17:57 -08:00
2021-02-10 23:34:16 +01:00
2021-03-04 06:36:22 -07:00
2021-02-12 15:27:17 +01:00
2021-02-21 10:29:20 -08:00
2021-02-12 15:27:17 +01:00
2021-02-23 11:29:52 -08:00
2021-02-05 20:48:28 +01:00
2021-02-08 12:24:04 +01:00
2021-02-24 13:38:31 -08:00
2021-02-26 09:41:03 -08:00
2021-02-17 14:12:42 +01:00
2021-02-26 10:19:03 -08:00
2021-02-22 10:22:54 -08:00
2021-02-26 09:41:02 -08:00
2021-02-26 09:41:05 -08:00
2021-02-26 09:40:59 -08:00
2021-02-26 09:41:03 -08:00
2021-02-22 13:16:53 -05:00
2021-02-19 11:54:08 +01:00
2021-02-19 11:54:08 +01:00
2021-02-19 11:35:28 +01:00
2021-02-23 13:39:45 -08:00
2021-02-23 13:39:45 -08:00
2021-02-26 09:41:03 -08:00
2021-02-09 09:30:16 +01:00
2021-02-22 13:01:23 -08:00
2021-02-24 13:38:30 -08:00
2021-02-26 09:41:00 -08:00
2021-02-26 09:41:00 -08:00
2021-02-26 09:41:00 -08:00
2021-02-24 13:38:35 -08:00
2021-02-24 13:38:33 -08:00
2021-02-24 13:38:32 -08:00
2021-02-26 09:41:00 -08:00
2021-02-24 10:25:37 -08:00
2021-02-08 12:28:07 +01:00
2021-02-27 08:07:12 -08:00
2021-02-10 14:44:40 +01:00
2021-02-16 19:35:29 -08:00
2021-02-23 20:32:11 -07:00
2021-02-11 13:24:44 -08:00
2021-03-09 18:07:05 -08:00
2021-02-16 16:11:14 -05:00
2021-02-26 09:17:24 -08:00
2021-02-12 19:23:57 -06:00
2021-02-11 08:33:41 +01:00
2021-02-17 13:17:49 -08:00
2021-02-24 13:38:29 -08:00
2021-02-26 09:41:01 -08:00
2021-02-26 09:40:59 -08:00
2021-02-26 09:40:59 -08:00
2021-02-25 09:56:08 -08:00
2021-02-23 14:11:13 -06:00
2021-02-23 14:12:01 -06:00
2021-02-23 09:28:51 -08:00
2021-02-26 09:41:05 -08:00
2021-02-16 15:23:23 -08:00
2021-02-24 14:52:41 +01:00
2021-02-05 10:08:10 +01:00
2021-03-12 17:09:34 -08:00
2021-02-26 09:41:03 -08:00
2021-02-17 14:07:44 +01:00
2021-02-21 12:35:04 -08:00
2021-02-26 09:41:01 -08:00
2021-03-09 18:07:05 -08:00
2021-02-23 13:39:45 -08:00
2021-03-14 14:48:02 -07:00
2021-02-26 12:28:04 -08:00
2021-02-26 09:41:02 -08:00
2021-02-26 09:41:02 -08:00
2021-03-10 12:34:01 -08:00
2021-02-26 09:41:04 -08:00
2021-02-17 14:12:42 +01:00
2021-02-17 14:12:42 +01:00
2021-02-17 14:25:50 -08:00
2021-02-26 09:41:04 -08:00
2021-03-02 17:25:46 -07:00
2021-02-20 10:13:32 -05:00
2021-02-23 13:39:45 -08:00
2021-03-01 15:25:24 -08:00
2021-02-06 00:21:42 +01:00
2021-02-16 10:40:28 +02:00
2021-03-04 09:51:50 -05:00
2021-02-09 12:27:29 -05:00
2021-02-20 21:28:04 -08:00
2021-02-13 17:17:53 +01:00
2021-02-04 18:56:56 -08:00
2021-02-15 18:50:01 +01:00
2021-02-23 07:52:56 -05:00
2021-03-09 16:12:20 -08:00
2021-02-23 07:52:58 -05:00
2021-02-12 12:56:55 +01:00
2021-02-09 12:15:07 +01:00
2021-02-26 09:41:00 -08:00
2021-02-26 09:41:01 -08:00
2021-02-26 09:41:01 -08:00
2021-02-08 22:58:55 +01:00
Alexander Lobakin
d206121faf