badsectoracula
today at 11:40 AM
The information about finetuning is interesting (it is something i'd like to do myself at some point, though i'll wait until i can do it with local hardware :-P). However FWIW LLMs are generally good at following a specific style when given examples.
As an example i asked Devstral Small 2 to write some docs for my LIL scripting language in the following style (this is copied from the DirectDraw documentation, edited to be text friendly):
IDirectDraw7::CreateClipper
---------------------------
The IDirectDraw7::CreateClipper method creates a DirectDrawClipper object.
HRESULT CreateClipper(
DWORD dwFlags,
LPDIRECTDRAWCLIPPER FAR *lplpDDClipper,
IUnknown FAR *pUnkOuter
);
Parameters
* dwFlags - Currently not used and must be set to 0.
* lplpDDClipper - Address of a variable to be set to a valid
IDirectDrawClipper interface pointer if the call succeeds.
* pUnkOuter - Allows for future compatibility with COM aggregation features.
Presently, however, this method returns an error if this parameter is
anything but NULL.
Return Values
If the method succeeds, the return value is DD_OK.
If it fails, the method can return one of the following error values:
* DDERR_INVALIDOBJECT
* DDERR_INVALIDPARAMS
* DDERR_NOCOOPERATIVELEVELSET
* DDERR_OUTOFMEMORY
Remarks
The DirectDrawClipper object can be attached to a DirectDrawSurface and used
during IDirectDrawSurface7::Blt, IDirectDrawSurface7::BltBatch, and
IDirectDrawSurface7::UpdateOverlay operations.
To create a DirectDrawClipper object that is not owned by a specific
DirectDraw object, use the DirectDrawCreateClipper function.
Requirements
Windows NT/2000: Requires Windows 2000.
Windows 95/98: Requires Windows 98.
Header: Declared in ddraw.h.
See Also
IDirectDrawSurface7::GetClipper, IDirectDrawSurface7::SetClipper
And it did a fine job. I put the full transcript in[0] to check out. The neat bit is that it can even handle weird formats like a custom documentation format i have (which only exists in my PC because i haven't released it anywhere) for a "master document" that can then be converted to various other file types. I gave it an example of some code in that and asked it to convert the documentation to it (this is part of the transcript at the end). Then i copy/pasted the generated code to a new file (adding a few extra lines the doc system expects which weren't part of the example - BTW i did not had to modify the generated code at all) and from that i generated a CHM file[1]. FWIW here is a comparison with the DirectX page i copied[2] (though consider that the generated pages went through the doc format which forces its own style and the textual output in the transcript matches the given style better).
[0] https://app.filen.io/#/d/9f4c1225-3527-4f16-a522-0678342120c...
[1] http://runtimeterror.com/pages/iv/images/45f8df428afe4fe6b6a...
[2] http://runtimeterror.com/pages/iv/images/ee58032790a049d7e74...
> However FWIW LLMs are generally good at following a specific style when given examples.
In your experience, is it worthwhile to have an agent create a "skill" for itself for following the style? Or is it a better use of context to just have it review the examples?
theletterf
today at 11:44 AM
Interesting! I think the advantage of style fine-tuning is that you might not have to provides that much context upfront. Also, it's kind of magical to have an LLM just do something out of the box. I'll compare my local fine-tuned models against the baseline with instructions and see how they fare.
badsectoracula
today at 11:51 AM
Yes, TBH one of the reasons i want to try and finetune my own is to teach it stuff that now i have to explain before anything can be done :-P.
Unfortunately i only have a 24GB GPU - and an AMD one at that - so there isn't much i can do on that front. Supposedly a 24GB GPU is enough for finetuning a 24B model with 4bit QLoRA, though when i tried it with some finetuning app (in an official docker container) it barfed at Mistral's weird template or something and i lost interest after that.
theletterf
today at 12:56 PM
Try Runpod or similar services: you can fine-tune stuff for the price of a latte. Stanford's NLP course recommends them: https://cs336.stanford.edu/
badsectoracula
today at 1:27 PM
I've heard about this but i prefer to avoid anything cloud based (not just for AI but in general). I try to avoid relying on stuff i have little to no control over.