Leaderboard
Subnet 9 - AI Pretraining

Subnet 9 is a Bittensor subnet that rewards miners for producing pretrained Foundation-Models on the Hugging Face's FineWeb Edu (score 2) dataset.

It acts like a continuous benchmark whereby miners are rewarded for attaining the best losses on randomly sampled pages.

The models with the best head-to-head win-rate on the evaluation data receive a steady emission of TAO.

Links:

Discord Github Whitepaper

Competitions

This is a Multi-Dataset competition.

Model improvement over time

Live Leaderboard

2nd

tensoralchemistde...

UID 158

Incentive0.0000Emission0.0000Trust0.0000Average Loss1.9721

1st

tensoralchemistde...

UID 148

Incentive0.0000Emission0.0000Trust0.0000Average Loss1.9654

3rd

deepnet111 / sn9-...

UID 5

Incentive0.0000Emission0.0000Trust0.0000Average Loss1.9698

Rank
1st	tensoralchemistdev01 / bb7	0.521	0.209	1.965	0.3564	0.67	-	3aca383e14d88406e8903cd0c1814102f16b42cf	148	4722185
2nd	tensoralchemistdev01 / bb1	0.24	0.096	1.972	0.1948	0.67	LlamaForCausalLM	74846487d29c0e7a31ee5b407577c6470c4dd7b2	158	4710063
3rd	deepnet111 / sn9-14b-star-001	0.239	0.096	1.97	0.3365	0.67	LlamaForCausalLM	281d7ece1dbaad9a1baf2968463a0a9d1d07902f	5	4720688
4	rwh / coder15	0	0	2.017	0.02	0	SlicedLlamaForCausalLM	e8c573d4609263c5a862aec2f11c5e31d960d47d	185	4637797
5	rwh / coder11	0	0	2.105	0.02	0.75	SlicedLlamaForCausalLM	90beed557c6ee90027e1dea7aaf3a60f4e6027a1	196	4518419
6	rwh / coder04	0	0	2.079	0.02	0.33	SlicedLlamaForCausalLM	cb024a2ee8e9d71b914202c37c0735e80cc9649b	203	4450296
7	jw-hf-test / jw-14bstar-002	0	0	1.849	0.3896	0.75	LlamaForCausalLM	663adfbf9b72519a1909366a9e585a503247d123	27	4486347
8	jw-hf-test / jw-14B-211	0	0	2.072	0.02	0	LlamaForCausalLM	21a676132f4bdf1a6d0970569d30c87a6bac5fdd	30	4039523
9	rwh / coder06	0	0	2.048	0.02	0.75	SlicedLlamaForCausalLM	e55937f13d463e3daf1770ad306086bed300bb97	206	4464734
10	rwh / coder08	0	0	1.996	0.02	0.75	SlicedLlamaForCausalLM	511833cec6c3ec0a80249000bc76f37cd4232190	190	4486579
11	rwh / coder03	0	0	2.056	0.02	0.33	SlicedLlamaForCausalLM	bf4e68a954cb25fccdb66609f2b4395b047e172f	199	4450033
12	visdata / bc18	0	0	1.964	0.41	0	LlamaForCausalLM	b4c16b6f0a355115ae9f45227df4bdfa5fae5d12	166	4711579
13	rwh / coder09	0	0	1.917	0.02	0.67	SlicedLlamaForCausalLM	907da752f308f7b1945c30e6063d886e27837382	193	4492260
14	jw-hf-test / jw-14B-300	0	0	2.16	0.02	0	LlamaForCausalLM	fa70fa9acabae2f7561bed154125d9b3aed3aeda	8	4646444
15	jw-hf-test / jw-14B-220	0	0	2.109	0.02	0.5	LlamaForCausalLM	81f19366fd0e164298ebf83c708038e153fc0556	25	4485603
16	jw-hf-test / jw-14B-219	0	0	2.103	0.02	0	LlamaForCausalLM	a8c5439c4c4fdd79127c019111df9e77f71ca8c6	69	4392751
17	axon2 / sn9_14b_106	0	0	1.894	0.3446	0	LlamaForCausalLM	27e763871be209c08fbd3b0b01187397c0feb4aa	106	4762719
18	rwh / h005	0	0	2.071	0.02	0	SlicedLlamaForCausalLM	b2bc15651e0bca57e4f8a2b0e57b289603b11469	202	4511390
19	jw-hf-test / jw-14B-218	0	0	2.021	0.02	0	LlamaForCausalLM	dd28a9d53955ea6aa794faa523c218c4323ee152	66	4291087
20	jw-hf-test / jw-14B-215	0	0	2.111	0.02	0	LlamaForCausalLM	62c495f181db4880336c15e29175304a52c06597	2	4199668
21	rwh / coder10	0	0	2.131	0.05	0	-	78a724e3eb02c00303c4b5330d50f480669f7f79	194	4494763
22	jw-hf-test / jw-14bstar-001	0	0	1.931	0.02	0	LlamaForCausalLM	0bcbe76da9691aa6d4409f9a4ba5fd2ad81e6d7b	114	4394129
23	jw-hf-test / jw-14B-214	0	0	2.11	0.02	0	LlamaForCausalLM	7f0ee237035437bc779d05e1672f87602f9f9303	16	4149650
24	jw-hf-test / jw-14B-216	0	0	2.131	0.02	0	LlamaForCausalLM	50c4ddc98c241e46c06bb594df413c0b74110b5e	252	4222047
25	deepnet111 / sn9-14b-star-005	0	0.4	2.018	0.02	0	LlamaForCausalLM	8924b4ec2cb93556bce6c80df77d864b3a5cfacf	24	4745155
26	visdata / bb_01	0	0	2.006	0.02	0	LlamaForCausalLM	c129dff0e0007f18effcc944746540cbeeb34228	82	4658280
27	deepnet111 / sn9-14b-star-002	0	0	1.953	0.4511	0	LlamaForCausalLM	ad5b75c770e6b81cce0d0ab7a5a69ad5da9ae66c	4	4728534
28	jw-hf-test / jw-14B-302	0	0	2.399	0.02	0	LlamaForCausalLM	1627663b233996b61de432940d3d3dd147662d48	80	4688895
29	rwh / coder12	0	0	2.082	0.05	0.25	-	b14f04f676b962a0205dd0fb2c2ea7ce1bacdcb8	188	4561324
30	rwh / huge8	0	0	2.131	0.02	0	SlicedLlamaForCausalLM	c2c9187de5451968bf8e91a10857ac7845bf93d2	179	4137216
31	deepnet / SN9-C4-llama-HK5-1	0	0	2.144	0.02	0.17	LlamaForCausalLM	fc51dd83c5634f8ddab2c6101e428674e5ae5397	127	4136683
32	rwh / coder16	0	0	2.154	0.05	0	-	30d77150c709c05f357a09acacf84e459c6a84be	183	4665639
33	rwh / h003	0	0	2.066	0.02	0	SlicedLlamaForCausalLM	129b59fa074d165618be6b8644711798b21dddbb	204	4450152
34	aipretrained / SN9_T1101FF06	0	0	2.079	0.02	0	LlamaForCausalLM	9753d7a5c018fbe7a9c600efccfa75e296817072	21	4057184
35	aipretrained / SN9_T11010004	0	0	2.118	0.05	0	LlamaForCausalLM	952bec4cb3ad192f379bfed51b72eb558182095c	58	4056834
36	visdata / quick5	0	0	2.104	0.4849	0	LlamaForCausalLM	29f56378a7c5c3c9548812ec0836d7361e5a1a2f	107	4733200
37	deepnet111 / sn9-14b-006	0	0	2.003	0.3453	0	LlamaForCausalLM	a8543f6ae53521bbfa37f2d9641ed6a58a72960c	12	4713040
38	rwh / multi01	0	0	2.054	0.05	0	-	5df4ef4c62edeb7cfe62bc04987aeed8e0ed7f3e	182	4725623
39	yonderjay / house-11-14-04	0	0	2.137	0.02	0	LlamaForCausalLM	21df74899d6dcad9e372a237b7110677f8e5b5b2	54	4271335
40	jw-hf-test / jw-14B-400	0	0	2.113	0.4897	0	LlamaForCausalLM	2b97ef776cb047eddabc8eed43dca2f2335adbc6	144	4701534
41	visdata / bbq1	0	0	2.086	0.4029	0	LlamaForCausalLM	94db96e6ae9dd98a3c76cc8de91ac6f5db9e6833	131	4703717
42	visdata / bbt1	0	0	2.077	0.4504	0	LlamaForCausalLM	4fc0ecebd81307624c627239a57c71c17993febc	129	4675201
43	jw-hf-test / jw-14B-221	0	0	2.09	0.3126	0	LlamaForCausalLM	6f7c2dae1bf9adc72a7db41c840090f837be5de2	88	4517485
44	luaqi / sn9_fat_v2	0	0	2.14	0.02	0	LlamaForCausalLM	4b626fad56b5a14da0cebb518454107007d6dd6e	221	3754511
45	jw-hf-test / jw-14B-303	0	0	2.076	0.3131	0	LlamaForCausalLM	c1dae6013a8d9b80db2fdbead69a58e0a2f96cb8	18	4695462
46	visdata / bbtz1	0	0	2.041	0.1616	0	LlamaForCausalLM	9fae4e3dbfeb3ff595f031ed33ee9c878726923c	96	4681560
47	visdata / quick25	0	0	2.109	0.4887	0	-	c7674eeeafa5780b95eaf6cc1deed4641ad69a84	71	4762027
48	visdata / quick24	0	0	2.094	0.4948	0	-	273358d36c59b2f8a90d28f7de963820d547a18d	102	4761920
49	visdata / quick20	0	0	2.004	0.4927	0	-	0e6a6ca509b8c575c4f311dc7cfc90351d38497c	121	4759333
50	axon2 / sn9_14b_119	0	0	2.072	0.4949	0	LlamaForCausalLM	692402b0c9146a603305edbd5e91aa99652ca5cb	119	4747050

Live Per-dataset Leaderboard

Daily Perplexity Benchmarks

Daily SOTA Benchmarks

No benchmarks available for this competition.

Leaderboard Subnet 9 - AI Pretraining