This is an AI translated post.
Google’s New Gemini Lineup - Experimental
- Writing language: Korean
- •
-
Base country: All countries
- •
- Information Technology
0000-00-00 00:00:00
Select Language
Summarized by durumis AI
- Google has released new experimental models, including Gemini 1.5 Pro Exp, Gemini 1.5 Flash Exp, and Gemini 1.5 Flash 8B Exp. In particular, Gemini 1.5 Pro Exp has improved performance compared to the previous version, and Gemini 1.5 Flash Exp has been upgraded to the best level among entry versions.
- Gemini 1.5 Flash 8B Exp shows performance similar to Claude 3 Sonnet and is a lightweight version of the Flash lineup. While it is somewhat lacking in translation of complex JSON structure text compared to the Pro lineup, it shows satisfactory results in summarization and writing tasks.
- Google is expected to offer Gemini 1.5 Flash 8B Exp at an affordable price when it is officially released, which could have a significant impact on other company lineups. We plan to share a price comparison analysis again after release.
Google recently released new models of Gemini.
Actually, they are not the official versions, but experimental versions.
The models are as follows:
- Gemini 1.5 Pro Exp
- Gemini 1.5 Flash Exp
- Gemini 1.5 Flash 8B Exp
First, Gemini 1.5 Pro Exp and 1.5 Flash Exp can be seen as updates to the existing versions. In fact, when tested, Gemini 1.5 Pro Exp showed a slight performance improvement compared to the existing 1.5 Pro (benchmark performance also improved. I didn't bring the data, but it can be considered the best among the existing released versions). Gemini 1.5 Flash Exp has also improved to the best level among entry-level versions, although not the best, of course.
These two versions will be reflected in the existing versions 1.5 Pro and 1.5 Flash within a few weeks. (Since it was version 001 before, it seems that it will be updated to version 002).
Source: Chatbot Arena
Checking the table, Gemini 1.5 Pro Exp is in 2nd place, and Gemini 1.5 Flash Exp is in 6th place.
Interestingly, Gemini 1.5 Flash Exp is ranked higher than Gemini 1.5 Pro, which is ranked 10th and 11th.
Ranks 1-5 are the flagship versions of each company (GPT 4o, Gemini 1.5 Pro, Grok 2), and ranks 6 and 7 are the entry-level lines of each company (GPT 4o mini, Gemini 1.5 Flash). Claude 3.5 Sonnet was at the top for a while... The development speed of this industry is truly remarkable...
Anyway, personally, I think these two versions (1.5 Pro, 1.5 Flash) will become official versions soon, and I'm currently curious about the 1.5 Flash 8b version.
Let's take a look at the benchmark first.
Source: Chatbot Arena
Looking at the benchmark above, the performance of 1.5 Flash 8b Exp is roughly similar to the existing Claude 3 Sonnet, and it's slightly lower than the existing 1.5 Flash, but shows similar performance to Llama 3 70b, etc.
Flash is a lightweight version of Pro, and Flash 8b appears to be a much more lightweight version than that. (Probably, right?)
First, I've done some direct testing, using sentence tests that we use a lot in our service.
- Translate.
- Summarize.
- Write text.
There are many other tests elsewhere, so I've briefly tested a few. When translating text with a complex JSON structure, Flash didn't produce satisfactory results, but the Pro lineup definitely did. Moreover, the Pro Exp version produced even cleaner results.
In summarization and text writing, both Flash and Flash 8b showed satisfactory results. It seems that Flash 8b can be utilized for simple tasks, as long as they are not highly complex tasks.
Considering the performance and parameters, it seems that Google will set a very low price for Flash 8b when it's officially released.
It's likely that they will set a price that will significantly damage the existing competitor's lineups. When it's released, I'll come back with the price list "again".
