opt-30b


Find this model in the OPT model summary

Model source: https://huggingface.co/facebook/opt-13b

opt-30b Model Summary Plots

opt-30b Model Selected Details
  layer_type N M Q alpha D alpha-hat log_SN rank_loss
layer_id                  
2 DENSE 7168 2050 3.50 4.31 0.10 8.52 1.98 1
4 DENSE 7168 7168 1.00 4.31 0.03 6.79 1.57 10
5 DENSE 7168 7168 1.00 6.20 0.05 4.62 0.75 10
6 DENSE 7168 7168 1.00 2.74 0.04 3.69 1.35 10
7 DENSE 7168 7168 1.00 4.19 0.07 10.15 2.43 15
9 DENSE 28672 7168 4.00 1.96 0.05 6.12 3.12 0
10 DENSE 28672 7168 4.00 4.15 0.05 8.90 2.15 0
12 DENSE 7168 7168 1.00 2.20 0.02 5.85 2.66 10
13 DENSE 7168 7168 1.00 4.18 0.04 6.48 1.55 11
14 DENSE 7168 7168 1.00 2.25 0.03 5.38 2.39 10
15 DENSE 7168 7168 1.00 3.65 0.05 8.28 2.27 15
17 DENSE 28672 7168 4.00 1.93 0.04 5.40 2.80 0
18 DENSE 28672 7168 4.00 4.03 0.04 9.47 2.35 0
20 DENSE 7168 7168 1.00 2.22 0.03 5.73 2.58 10
21 DENSE 7168 7168 1.00 3.99 0.05 6.15 1.54 11
22 DENSE 7168 7168 1.00 2.16 0.03 5.13 2.38 9
23 DENSE 7168 7168 1.00 3.70 0.05 7.96 2.15 14
25 DENSE 28672 7168 4.00 2.39 0.04 6.63 2.78 0
26 DENSE 28672 7168 4.00 4.16 0.04 10.05 2.42 0
28 DENSE 7168 7168 1.00 2.19 0.03 5.66 2.58 11
29 DENSE 7168 7168 1.00 4.32 0.05 5.34 1.24 10
30 DENSE 7168 7168 1.00 2.27 0.02 5.31 2.34 11
31 DENSE 7168 7168 1.00 3.77 0.04 7.33 1.94 13
33 DENSE 28672 7168 4.00 2.37 0.03 6.55 2.76 0
34 DENSE 28672 7168 4.00 4.43 0.03 11.01 2.49 0
36 DENSE 7168 7168 1.00 2.24 0.03 5.82 2.60 12
37 DENSE 7168 7168 1.00 4.36 0.04 5.29 1.22 11
38 DENSE 7168 7168 1.00 2.33 0.02 5.32 2.28 11
39 DENSE 7168 7168 1.00 3.84 0.04 7.26 1.89 12
41 DENSE 28672 7168 4.00 2.35 0.02 6.49 2.76 0
42 DENSE 28672 7168 4.00 4.39 0.03 11.07 2.52 0
44 DENSE 7168 7168 1.00 2.25 0.03 5.76 2.56 10
45 DENSE 7168 7168 1.00 4.62 0.04 5.65 1.22 11
46 DENSE 7168 7168 1.00 2.30 0.02 5.28 2.30 12
47 DENSE 7168 7168 1.00 3.80 0.04 6.72 1.77 13
49 DENSE 28672 7168 4.00 2.35 0.02 6.44 2.74 0
50 DENSE 28672 7168 4.00 4.38 0.03 11.30 2.58 0
52 DENSE 7168 7168 1.00 2.19 0.02 5.57 2.55 11
53 DENSE 7168 7168 1.00 4.49 0.03 5.57 1.24 9
54 DENSE 7168 7168 1.00 2.24 0.01 5.07 2.27 11
55 DENSE 7168 7168 1.00 3.81 0.04 6.38 1.67 11
57 DENSE 28672 7168 4.00 2.34 0.01 6.38 2.73 0
58 DENSE 28672 7168 4.00 4.18 0.02 10.83 2.59 0
60 DENSE 7168 7168 1.00 2.39 0.03 5.94 2.49 11
61 DENSE 7168 7168 1.00 4.44 0.03 5.47 1.23 12
62 DENSE 7168 7168 1.00 2.42 0.03 5.51 2.27 12
63 DENSE 7168 7168 1.00 3.68 0.04 6.09 1.65 12
65 DENSE 28672 7168 4.00 2.36 0.01 6.40 2.71 0
66 DENSE 28672 7168 4.00 4.11 0.01 10.66 2.59 0
68 DENSE 7168 7168 1.00 2.38 0.02 5.78 2.43 11
69 DENSE 7168 7168 1.00 4.13 0.02 5.15 1.25 10
70 DENSE 7168 7168 1.00 2.43 0.02 5.44 2.24 11
71 DENSE 7168 7168 1.00 3.52 0.03 5.86 1.67 9
73 DENSE 28672 7168 4.00 2.38 0.01 6.45 2.71 0
74 DENSE 28672 7168 4.00 4.13 0.01 10.70 2.59 0
76 DENSE 7168 7168 1.00 2.31 0.02 5.51 2.39 10
77 DENSE 7168 7168 1.00 4.01 0.01 4.93 1.23 10
78 DENSE 7168 7168 1.00 2.29 0.02 5.06 2.21 12
79 DENSE 7168 7168 1.00 3.45 0.03 5.50 1.59 11
81 DENSE 28672 7168 4.00 2.40 0.01 6.53 2.72 0
82 DENSE 28672 7168 4.00 4.22 0.01 10.86 2.57 0
84 DENSE 7168 7168 1.00 2.54 0.02 5.98 2.35 12
85 DENSE 7168 7168 1.00 3.92 0.01 5.06 1.29 9
86 DENSE 7168 7168 1.00 2.43 0.02 5.18 2.13 11
87 DENSE 7168 7168 1.00 3.36 0.02 5.17 1.54 12
89 DENSE 28672 7168 4.00 2.42 0.01 6.68 2.76 0
90 DENSE 28672 7168 4.00 4.26 0.01 10.86 2.55 0
92 DENSE 7168 7168 1.00 1.91 0.02 4.27 2.23 12
93 DENSE 7168 7168 1.00 3.71 0.02 4.48 1.21 9
94 DENSE 7168 7168 1.00 2.46 0.02 5.24 2.13 11
95 DENSE 7168 7168 1.00 3.30 0.01 5.07 1.54 11
97 DENSE 28672 7168 4.00 2.42 0.01 6.75 2.79 0
98 DENSE 28672 7168 4.00 4.51 0.01 11.40 2.53 0
100 DENSE 7168 7168 1.00 1.88 0.02 4.12 2.19 11
101 DENSE 7168 7168 1.00 3.77 0.02 4.47 1.18 9
102 DENSE 7168 7168 1.00 2.34 0.02 4.87 2.08 9
103 DENSE 7168 7168 1.00 3.11 0.01 4.18 1.35 10
105 DENSE 28672 7168 4.00 2.43 0.01 6.93 2.86 0
106 DENSE 28672 7168 4.00 4.79 0.02 11.62 2.42 0
108 DENSE 7168 7168 1.00 2.59 0.02 5.68 2.20 10
109 DENSE 7168 7168 1.00 3.68 0.02 4.31 1.17 9
110 DENSE 7168 7168 1.00 2.35 0.02 4.98 2.12 10
111 DENSE 7168 7168 1.00 3.19 0.02 3.76 1.18 11
113 DENSE 28672 7168 4.00 2.44 0.01 6.98 2.86 0
114 DENSE 28672 7168 4.00 4.71 0.02 11.17 2.37 0
116 DENSE 7168 7168 1.00 2.49 0.02 5.33 2.14 10
117 DENSE 7168 7168 1.00 3.53 0.02 4.11 1.16 8
118 DENSE 7168 7168 1.00 2.31 0.02 4.92 2.13 9
119 DENSE 7168 7168 1.00 2.99 0.02 3.23 1.08 11
121 DENSE 28672 7168 4.00 2.48 0.01 7.06 2.85 0
122 DENSE 28672 7168 4.00 4.69 0.02 10.40 2.22 0
124 DENSE 7168 7168 1.00 2.72 0.02 6.39 2.35 9
125 DENSE 7168 7168 1.00 3.53 0.01 4.48 1.27 10
126 DENSE 7168 7168 1.00 2.48 0.02 5.54 2.23 9
127 DENSE 7168 7168 1.00 3.02 0.02 3.12 1.03 10
129 DENSE 28672 7168 4.00 2.51 0.01 6.94 2.77 0
130 DENSE 28672 7168 4.00 4.26 0.01 8.95 2.10 0
132 DENSE 7168 7168 1.00 2.53 0.01 5.75 2.27 11
133 DENSE 7168 7168 1.00 3.45 0.01 3.92 1.14 9
134 DENSE 7168 7168 1.00 2.38 0.01 5.31 2.23 9
135 DENSE 7168 7168 1.00 3.11 0.02 3.26 1.05 10
137 DENSE 28672 7168 4.00 2.55 0.01 6.88 2.70 0
138 DENSE 28672 7168 4.00 4.07 0.01 8.38 2.06 0
140 DENSE 7168 7168 1.00 2.52 0.01 5.98 2.37 11
141 DENSE 7168 7168 1.00 3.55 0.01 4.45 1.26 8
142 DENSE 7168 7168 1.00 2.36 0.01 5.45 2.31 9
143 DENSE 7168 7168 1.00 3.22 0.02 3.17 0.98 10
145 DENSE 28672 7168 4.00 2.55 0.01 6.70 2.62 0
146 DENSE 28672 7168 4.00 4.04 0.01 8.16 2.02 0
148 DENSE 7168 7168 1.00 2.56 0.01 6.19 2.42 10
149 DENSE 7168 7168 1.00 3.43 0.01 4.08 1.19 9
150 DENSE 7168 7168 1.00 2.42 0.01 5.56 2.30 8
151 DENSE 7168 7168 1.00 3.00 0.04 3.19 1.06 7
153 DENSE 28672 7168 4.00 2.45 0.01 6.37 2.60 0
154 DENSE 28672 7168 4.00 3.65 0.01 7.49 2.05 0
156 DENSE 7168 7168 1.00 2.47 0.01 5.87 2.38 10
157 DENSE 7168 7168 1.00 3.85 0.01 4.25 1.10 9
158 DENSE 7168 7168 1.00 2.35 0.01 5.17 2.20 9
159 DENSE 7168 7168 1.00 3.69 0.04 3.72 1.01 9
161 DENSE 28672 7168 4.00 2.44 0.02 6.37 2.61 0
162 DENSE 28672 7168 4.00 3.65 0.01 7.46 2.04 0
164 DENSE 7168 7168 1.00 2.51 0.01 6.04 2.41 8
165 DENSE 7168 7168 1.00 3.74 0.01 4.29 1.15 8
166 DENSE 7168 7168 1.00 2.36 0.01 5.05 2.14 8
167 DENSE 7168 7168 1.00 3.52 0.02 3.93 1.12 10
169 DENSE 28672 7168 4.00 2.54 0.01 6.64 2.62 0
170 DENSE 28672 7168 4.00 3.51 0.01 7.16 2.04 0
172 DENSE 7168 7168 1.00 2.65 0.02 6.24 2.35 8
173 DENSE 7168 7168 1.00 3.99 0.02 4.93 1.23 7
174 DENSE 7168 7168 1.00 2.48 0.02 5.07 2.04 9
175 DENSE 7168 7168 1.00 4.12 0.03 4.52 1.10 8
177 DENSE 28672 7168 4.00 2.57 0.01 6.75 2.63 0
178 DENSE 28672 7168 4.00 3.32 0.01 6.61 1.99 0
180 DENSE 7168 7168 1.00 2.53 0.02 5.94 2.35 8
181 DENSE 7168 7168 1.00 3.93 0.02 4.84 1.23 9
182 DENSE 7168 7168 1.00 2.28 0.02 4.61 2.02 7
183 DENSE 7168 7168 1.00 4.65 0.03 5.43 1.17 9
185 DENSE 28672 7168 4.00 2.61 0.01 6.91 2.65 0
186 DENSE 28672 7168 4.00 3.12 0.01 6.15 1.97 0
188 DENSE 7168 7168 1.00 2.66 0.02 6.21 2.33 8
189 DENSE 7168 7168 1.00 3.98 0.02 4.57 1.15 7
190 DENSE 7168 7168 1.00 2.37 0.02 4.67 1.97 7
191 DENSE 7168 7168 1.00 3.93 0.04 4.16 1.06 8
193 DENSE 28672 7168 4.00 2.64 0.01 7.02 2.65 0
194 DENSE 28672 7168 4.00 3.00 0.01 5.96 1.99 0
196 DENSE 7168 7168 1.00 2.70 0.01 6.14 2.27 8
197 DENSE 7168 7168 1.00 3.73 0.03 4.42 1.19 8
198 DENSE 7168 7168 1.00 2.32 0.03 4.46 1.92 8
199 DENSE 7168 7168 1.00 4.26 0.04 4.74 1.11 8
201 DENSE 28672 7168 4.00 2.64 0.01 7.05 2.67 0
202 DENSE 28672 7168 4.00 2.98 0.01 5.91 1.98 0
204 DENSE 7168 7168 1.00 2.64 0.01 6.11 2.32 8
205 DENSE 7168 7168 1.00 3.52 0.03 4.20 1.19 9
206 DENSE 7168 7168 1.00 2.28 0.02 4.41 1.93 7
207 DENSE 7168 7168 1.00 3.55 0.05 3.67 1.03 8
209 DENSE 28672 7168 4.00 2.63 0.01 7.09 2.70 0
210 DENSE 28672 7168 4.00 2.96 0.01 5.86 1.98 0
212 DENSE 7168 7168 1.00 2.64 0.01 6.05 2.29 8
213 DENSE 7168 7168 1.00 3.41 0.04 3.98 1.17 8
214 DENSE 7168 7168 1.00 2.34 0.03 4.42 1.89 8
215 DENSE 7168 7168 1.00 6.40 0.05 6.72 1.05 8
217 DENSE 28672 7168 4.00 2.62 0.01 7.10 2.71 0
218 DENSE 28672 7168 4.00 2.94 0.01 5.78 1.97 0
220 DENSE 7168 7168 1.00 2.64 0.01 5.94 2.25 7
221 DENSE 7168 7168 1.00 3.29 0.03 4.02 1.22 8
222 DENSE 7168 7168 1.00 2.34 0.03 4.26 1.82 8
223 DENSE 7168 7168 1.00 5.36 0.04 5.82 1.09 9
225 DENSE 28672 7168 4.00 2.62 0.01 7.16 2.74 0
226 DENSE 28672 7168 4.00 2.99 0.01 5.96 1.99 0
228 DENSE 7168 7168 1.00 2.90 0.02 6.34 2.18 7
229 DENSE 7168 7168 1.00 3.11 0.04 3.70 1.19 8
230 DENSE 7168 7168 1.00 2.48 0.04 4.47 1.80 8
231 DENSE 7168 7168 1.00 3.22 0.06 3.36 1.04 7
233 DENSE 28672 7168 4.00 2.65 0.01 7.29 2.75 0
234 DENSE 28672 7168 4.00 2.99 0.01 5.81 1.94 0
236 DENSE 7168 7168 1.00 2.81 0.01 6.12 2.18 8
237 DENSE 7168 7168 1.00 3.05 0.03 3.91 1.28 8
238 DENSE 7168 7168 1.00 2.72 0.03 4.88 1.79 8
239 DENSE 7168 7168 1.00 3.24 0.05 3.66 1.13 10
241 DENSE 28672 7168 4.00 2.66 0.01 7.40 2.78 0
242 DENSE 28672 7168 4.00 3.09 0.01 5.80 1.87 0
244 DENSE 7168 7168 1.00 3.00 0.02 6.19 2.06 6
245 DENSE 7168 7168 1.00 3.02 0.04 3.81 1.26 6
246 DENSE 7168 7168 1.00 3.05 0.03 5.43 1.78 7
247 DENSE 7168 7168 1.00 2.95 0.05 3.26 1.11 8
249 DENSE 28672 7168 4.00 2.67 0.02 7.44 2.78 0
250 DENSE 28672 7168 4.00 3.05 0.01 5.75 1.89 0
252 DENSE 7168 7168 1.00 2.79 0.01 5.98 2.14 7
253 DENSE 7168 7168 1.00 2.97 0.03 3.85 1.30 9
254 DENSE 7168 7168 1.00 2.67 0.03 4.67 1.75 8
255 DENSE 7168 7168 1.00 2.94 0.06 3.18 1.08 8
257 DENSE 28672 7168 4.00 2.69 0.01 7.54 2.80 0
258 DENSE 28672 7168 4.00 3.12 0.01 5.70 1.83 0
260 DENSE 7168 7168 1.00 3.00 0.02 5.96 1.99 7
261 DENSE 7168 7168 1.00 3.28 0.05 4.21 1.28 7
262 DENSE 7168 7168 1.00 2.88 0.03 4.99 1.73 8
263 DENSE 7168 7168 1.00 3.04 0.05 3.49 1.15 8
265 DENSE 28672 7168 4.00 2.71 0.02 7.59 2.80 0
266 DENSE 28672 7168 4.00 3.04 0.01 5.58 1.83 0
268 DENSE 7168 7168 1.00 2.81 0.01 5.62 2.00 6
269 DENSE 7168 7168 1.00 3.27 0.05 4.05 1.24 7
270 DENSE 7168 7168 1.00 2.63 0.03 4.60 1.75 9
271 DENSE 7168 7168 1.00 2.96 0.05 3.44 1.16 8
273 DENSE 28672 7168 4.00 2.74 0.02 7.68 2.80 0
274 DENSE 28672 7168 4.00 2.99 0.01 5.59 1.87 0
276 DENSE 7168 7168 1.00 2.90 0.01 5.92 2.04 7
277 DENSE 7168 7168 1.00 4.03 0.04 4.67 1.16 8
278 DENSE 7168 7168 1.00 2.83 0.02 4.81 1.70 7
279 DENSE 7168 7168 1.00 8.52 0.04 9.85 1.16 8
281 DENSE 28672 7168 4.00 2.75 0.02 7.75 2.82 0
282 DENSE 28672 7168 4.00 3.06 0.01 5.74 1.88 0
284 DENSE 7168 7168 1.00 3.01 0.01 5.84 1.94 6
285 DENSE 7168 7168 1.00 4.18 0.05 4.60 1.10 7
286 DENSE 7168 7168 1.00 2.77 0.03 4.82 1.74 8
287 DENSE 7168 7168 1.00 7.58 0.03 9.17 1.21 8
289 DENSE 28672 7168 4.00 2.76 0.02 7.82 2.84 0
290 DENSE 28672 7168 4.00 3.08 0.02 5.85 1.90 0
292 DENSE 7168 7168 1.00 3.02 0.01 6.09 2.01 6
293 DENSE 7168 7168 1.00 6.76 0.04 7.94 1.17 7
294 DENSE 7168 7168 1.00 2.95 0.03 5.24 1.78 6
295 DENSE 7168 7168 1.00 6.75 0.03 8.24 1.22 8
297 DENSE 28672 7168 4.00 2.81 0.02 7.97 2.84 0
298 DENSE 28672 7168 4.00 3.19 0.02 5.97 1.87 0
300 DENSE 7168 7168 1.00 3.25 0.02 6.32 1.94 7
301 DENSE 7168 7168 1.00 5.20 0.04 6.18 1.19 8
302 DENSE 7168 7168 1.00 3.06 0.02 5.36 1.75 7
303 DENSE 7168 7168 1.00 6.43 0.04 8.02 1.25 6
305 DENSE 28672 7168 4.00 2.84 0.01 8.08 2.84 0
306 DENSE 28672 7168 4.00 3.28 0.02 6.10 1.86 0
308 DENSE 7168 7168 1.00 3.20 0.02 6.35 1.99 7
309 DENSE 7168 7168 1.00 6.98 0.03 7.88 1.13 7
310 DENSE 7168 7168 1.00 3.05 0.01 5.65 1.85 8
311 DENSE 7168 7168 1.00 4.99 0.02 6.39 1.28 7
313 DENSE 28672 7168 4.00 2.86 0.01 8.15 2.85 0
314 DENSE 28672 7168 4.00 3.48 0.03 6.50 1.87 0
316 DENSE 7168 7168 1.00 3.24 0.02 6.48 2.00 7
317 DENSE 7168 7168 1.00 8.56 0.04 10.31 1.21 6
318 DENSE 7168 7168 1.00 3.07 0.01 5.49 1.79 7
319 DENSE 7168 7168 1.00 6.32 0.02 8.07 1.28 6
321 DENSE 28672 7168 4.00 2.93 0.02 8.35 2.85 0
322 DENSE 28672 7168 4.00 3.82 0.02 7.15 1.87 0
324 DENSE 7168 7168 1.00 2.98 0.02 6.00 2.01 7
325 DENSE 7168 7168 1.00 8.12 0.05 8.57 1.06 6
326 DENSE 7168 7168 1.00 3.13 0.01 5.73 1.83 7
327 DENSE 7168 7168 1.00 7.80 0.03 9.91 1.27 7
329 DENSE 28672 7168 4.00 2.93 0.02 8.37 2.86 0
330 DENSE 28672 7168 4.00 3.97 0.02 7.24 1.83 0
332 DENSE 7168 7168 1.00 2.82 0.03 5.87 2.08 7
333 DENSE 7168 7168 1.00 9.15 0.05 10.13 1.11 7
334 DENSE 7168 7168 1.00 3.21 0.01 6.10 1.90 7
335 DENSE 7168 7168 1.00 6.93 0.01 8.43 1.22 6
337 DENSE 28672 7168 4.00 2.96 0.02 8.42 2.85 0
338 DENSE 28672 7168 4.00 3.95 0.01 7.04 1.78 0
340 DENSE 7168 7168 1.00 3.78 0.03 8.04 2.13 8
341 DENSE 7168 7168 1.00 9.85 0.04 11.00 1.12 8
342 DENSE 7168 7168 1.00 3.53 0.02 6.85 1.94 7
343 DENSE 7168 7168 1.00 7.48 0.02 8.84 1.18 6
345 DENSE 28672 7168 4.00 3.01 0.02 8.57 2.85 0
346 DENSE 28672 7168 4.00 3.88 0.01 7.07 1.82 0
348 DENSE 7168 7168 1.00 3.65 0.03 8.47 2.32 6
349 DENSE 7168 7168 1.00 9.97 0.04 11.29 1.13 6
350 DENSE 7168 7168 1.00 3.11 0.01 6.59 2.12 6
351 DENSE 7168 7168 1.00 6.88 0.02 7.97 1.16 7
353 DENSE 28672 7168 4.00 3.06 0.02 8.65 2.83 0
354 DENSE 28672 7168 4.00 3.88 0.01 7.20 1.86 0
356 DENSE 7168 7168 1.00 3.60 0.03 8.51 2.37 7
357 DENSE 7168 7168 1.00 14.07 0.07 14.60 1.04 6
358 DENSE 7168 7168 1.00 2.88 0.02 6.20 2.15 5
359 DENSE 7168 7168 1.00 9.52 0.07 11.61 1.22 7
361 DENSE 28672 7168 4.00 3.10 0.02 8.76 2.82 0
362 DENSE 28672 7168 4.00 3.95 0.01 7.63 1.93 0
364 DENSE 7168 7168 1.00 3.83 0.03 9.09 2.38 7
365 DENSE 7168 7168 1.00 11.90 0.03 12.77 1.07 6
366 DENSE 7168 7168 1.00 3.49 0.02 7.67 2.20 8
367 DENSE 7168 7168 1.00 7.48 0.06 11.53 1.54 6
369 DENSE 28672 7168 4.00 3.19 0.02 9.02 2.82 0
370 DENSE 28672 7168 4.00 3.89 0.02 7.93 2.04 0
372 DENSE 7168 7168 1.00 3.75 0.03 9.22 2.46 8
373 DENSE 7168 7168 1.00 8.13 0.05 10.22 1.26 5
374 DENSE 7168 7168 1.00 3.35 0.03 7.50 2.24 7
375 DENSE 7168 7168 1.00 6.19 0.07 10.87 1.76 7
377 DENSE 28672 7168 4.00 3.92 0.03 11.15 2.84 0
378 DENSE 28672 7168 4.00 4.12 0.02 9.01 2.19 0
380 DENSE 7168 7168 1.00 3.49 0.02 9.23 2.64 7
381 DENSE 7168 7168 1.00 6.83 0.05 8.97 1.31 7
382 DENSE 7168 7168 1.00 3.04 0.03 6.94 2.28 6
383 DENSE 7168 7168 1.00 5.16 0.08 8.40 1.63 9
385 DENSE 28672 7168 4.00 3.75 0.01 11.22 2.99 0
386 DENSE 28672 7168 4.00 3.93 0.03 8.97 2.28 0