Instruction: What is the recommended method for organizing and keeping track of research papers, and how can one find and read new interesting papers in their field of study?

Response: Don't forget to hold onto your papers. Bookmark and organize them because you'll have at least a 1000 saved one day and thank yourself if you took the time to make them easy to find. To find new interesting ones go to Google Scholar, look up your favorites and click 'cited by' to find new ones or go look at what new papers the authors have published. And of course read any interesting ones you find cited. Aim to read about 1-3 per day.


Instruction: Can you provide advice on how to convince someone who is skeptical about volunteer computing projects and believes that most unused power is on computers with a "give me that" mentality?

Response: You don't have to convince me Anon, I'm already 'part of the choir' with you. OTOH, how do you convince this anon? While I wish his point was entirely invalid, the simple fact is he's right. The vast majority of unused power out there resides on anon's computers who have grown accustomed to a 'gibbs me dat' mentality (not that I'm impugning him in any way in this regards, he's simply pointing the issue out). OTO-OH, many of these X@home projects do succeed at attracting numerous volunteer contributors -- even up to 100'000 of them.


Instruction: Can you explain the proposed idea for a multistable LSTM and how it differs from a monostable LSTM?

Response:
Busy working on other stuff at the moment but I'll leave my ideas here while they're fresh in my mind. Here's the idea I have for a monostable LSTM that introduces new layers of weights Us, Uq, Ws and Wq and a decay parameter, d.

s_t is the equilibrium set gate
~q_t is the value to set the equilibrium value
q_t is the current equilibrium value
C_t is the changed LSTM cell value

It should be possible to implement a simplistic multistable LSTM by setting a phase and frequency instead of an equilibrium value and shifting cell states that are outside of stable states by decay * sin(frequency * x*2*pi - phase*2*pi).
When frequency is 1 and phase is 0, the values will settle over time to a bistable state of -0.5 or 0.5.
When frequency is 1 and phase is 0.1, the bistable states will become -0.4 and 0.6.
When frequency is 2 and phase is 0, the multistable states will become -0.75, -0.25, 0.25, 0.75.

Decay could also be rewritten from being a parameter to being neuromodulated decay. Perhaps a non-linear function could be introduced somehow to allow arbitrary positions of the stable states, so it could have something like 0.1 and 0.7 without there being a -0.5. It'll be a fun idea to experiment with.


Instruction: What is the potential problem that can arise when adding flavor bits to TT's output, and how can it be avoided? What are some guidelines for adding flavor bits to TT's sentences without changing their original meaning?

Response: One problematic case is that the intended flavor bits completely change the meaning of the AI's output from something reasonable to nonsense. I think avoiding that in almost all situations is pretty easy: Just keep the flavor bits short and don't add negations to TT's sentences. Putting the flavor bits only before or after the TT output is far less risky than making multiple insertions right into its sentences.


Instruction: Can you provide more information on how to do parameter-efficient finetuning?

Response: There's now a PyTorch framework that makes parameter-efficient finetuning super simple and painless to do for any model called OpenDelta.


Instruction: Can you explain why a 16GB high-end GPU might be incompatible with certain tasks, and why the price of a low-end 4GB nvidia GPU has increased significantly while similarly specced or more powerful AMD cards haven't?

Response: Even if they have a 16GB high end GPU it might be incompatible with some tasks as AMD uses a different memory structure that can't run many popular CUDA compute libraries. That's why my low end 4GB nvidia GPU almost tripled in price in the last year while similarly specced or slightly more powerful AMD cards haven't.


Instruction: What skills are needed to contribute to the robowaifu project, and how can individuals with those skills be encouraged to get involved?

Response: Personally, I can program in C++ a bit, but don't really have much else to contribute to this specific project afaict. Not that I'm unwilling to help out with it, but I can't take more on my plate at the moment to learn whole other sets of skills. I'm not sure I can see that being an application developer would bring much to the table here -- please correct me if I'm wrong. So, in that sense I'm basically just another anon who can contribute a modest amount of compute cycles from my computers. But again, there are plenty of others who do have the needed skills who come through occasionally. Getting them interested could be key here.


Instruction: What is the Levenshtein edit distance and how does it help in adjusting the network weights to insert another character?

Response:
Small errors typically completely throw off training language models and make them difficult to train (see the image to see what I mean). With the Levenshtein edit distance though it only has to adjust the network weights to insert another character.

Calculating the edit distance is quite slow for long sequences, especially with a naive implementation it's O(n*m).


Instruction: What is the current state of understanding regarding the effectiveness of hypernets, and why is there ambiguity surrounding their definition?

Response: There's a lot of ambiguity over what gets called a "hypernet", and at least some variants end up being useless. From what I've heard, it isn't clear right now if hypernets are actually a good idea.


Instruction: Can you provide an estimate of the time it would take to perform a distributed reduce operation on 125MB of parameter update data per batch with internet latencies, assuming 10-100 people are training?

Response: My understanding is that that would still require 1 bit per parameter per batch, which for a 1B model would be 125MB of parameter update data per batch. How long would a distributed reduce operation take on that much data with Internet latencies? Assuming there are 10-100 people training, I would guess it would be on the order of minutes, given that upload speeds tend be much slower than download speeds.


Instruction: What are some techniques for reducing the size of models so they can run on mobile devices, and how do you anticipate this process will change in the near future?

Response: There's a lot of techniques for reducing models by 2-3 orders of magnitude with little accuracy loss so they can run on mobile devices, such as network pruning, sparse networks and knowledge distillation. Doing it manually today is quite a bit of work but it will be automated in the near future and consumer hardware and algorithms will be much faster, so I haven't been too worried about the runtime performance. But now that I think about it I'm sure people will want the latest and greatest features and demand maximum performance. We definitely don't want people buying Alexa spydroids because our damn GNU/Waifus run like fucking GIMP.


Instruction: Can you explain the benefits of interpolating a finetuned CLIP checkpoint with its starting checkpoint and how it improves performance in-distribution and out-of-distribution?

Response:
Basically, interpolating a finetuned CLIP checkpoint with its starting checkpoint improves its performance in-distribution and out-of-distribution.

A similar phenomenon is seen in lo-fi where the low-performing merged models outpace the baseline when merged together, although it performed slightly worse on language modelling. I believe this is due to how weights get updated in attention layers. I haven't investigated this thoroughly yet but a single backward pass is usually enough for a transformer to remember something and then those weights are rarely ever touched again. I've inspected text encoders of various Stable Diffusion models and often 70-95% of the parameters are exactly the same. So what nodes learn from mutually exclusive data will effectively be washed out when merged together.


Instruction: Can you describe how to identify a bot in a conversation and what are the common patterns in their responses that can give them away?

Response: I'm 99% certain they already exist. From talking with different language models so much I've become familiar the patterns in their responses. The most noticeable thing when you're talking to a bot is it will only respond to the past 500-1000 words or so, or only the page's content. If you drop a pdf file or link, it will talk about it pretending to know what its contents are. The most obvious thing is if you add meaningless sentences or paragraphs to your post of a completely unrelated topic, it will generate text in response to it.


Instruction: What kind of ethical guidelines or regulations should be put in place to prevent the creation and use of such traumatizing videos with humanoid robots?

Response: Imagine a humanoid robot sitting down during a protest lighting itself on fire, jumping off a tall structure, being crushed by a vehicle while being filmed by hundreds of people. Combine that with robots meant to appear as children that can bleed and wail profusely you've got a very powerful tool for traumatizing an audience with videos that can go viral. By the time the truth comes out(or if with state actors involved) it'll be irrelevant to the general public.


Instruction: What is being described as the preferred choice for programming and why do developers of Tensorflow, Torch, and other groups agree with this perspective?

Response: Not only are these systems generic, they are robust. Couple this with the near-optimal performance characteristics & the child's building-blocks like design strategy possible. I can hardly understand why anyone would program in anything else if you have to do any 'heavy lifting'. I expect developers of Tensorflow, Torch, and dozens of others groups also agree with that perspective, since they all used it to build the libraries themselves.


Instruction: What are some potential functions that people might want their virtual waifus to perform, and how could these functions be developed and made available to users?

Response: A virtual waifu and all her functions. Once basic chat is solved people are going to expand their virtual waifus to perform other functions such as playing video games, composing music, drawing, debating, summarizing research papers, searching the web, etc. People wanting these functions will contribute to those projects and receive a compressed version of the training results that their hardware can run or the full size version if they wish. Alternatively, someone could create a marketplace where people can pay crypto for compute, but I'm not familiar with how to do that. I think SingularityNET does something like that with AI services.


Instruction: Can you explain the potential benefits of using a pre-trained model, as mentioned in the passage, and the difficulties that may arise from using Anaconda as a Python environment, as suggested by the author? Additionally, can you discuss ways to troubleshoot issues related to Anaconda and Python incompatibilities?

Response: Yeah, a pre-trained model is probably best. BTW, I've been playing around with trying to figure out why Python is breaking and apparently the requirement of using Anaconda is causing at least some of the issues for me. It's some type of 'python VM environment' or something, apparently. Know anything about it Anon?


Instruction: What is non-monotonic logic, and how does it differ from monotonic logic? Can you provide an example to illustrate the difference between the two types of logic and how non-monotonic logic can lead to a shrinkage of conclusions?

Response: Non-monotonic logic can be explained by what monotonic logic is: In monotonic logic, you have a pile of assumptions and a pile of conclusions you generate from your assumptions. As you are adding new things to your pile of assumptions, your pile of conclusions can only grow. In monotonic logic, an established conclusion is not thrown out unless you delete something in your assumption pile that was necessary for generating the conclusion. In non-monotonic logic, new assumptions can add qualifications to other assumptions, so adding new assumptions can shrink your pile of conclusions. For example, you can have an old assumption that birds can fly and then add a new assumption that penguins are birds that can't fly.


Instruction: What does "it" refer to in this context, and how does it try to determine the relationship between two people? Does it have a theory of mind, and what limitations does it have in this regard?

Response: When there's a lack of direct statements about whether two people are friends, it still tries to figure out how one person might feel about a good or bad thing happening to another person by looking for indirect social relations between the pair; and if doing that doesn't help, it tries guessing by looking for data about the two having similar attitudes about some issues (page 189). It of course does not have as much common sense as even a human tween and it occasionally craps itself in the given examples, but I'm willing to say it does have a bit of a theory of mind for doing this.


Instruction: What considerations should be taken into account when designing a program that utilizes people's computers for prototyping models and how can feedback help ensure their continued participation?

Response:
What kind of instruction could the following be the answer to? Respond with a question, starting with "Instruction:"

Yea, some thought is gonna have to go into how to utilize people's computers effectively and automatically. They might only have a 2 GB toaster GPU. While not ideal it could still help prototyping smaller models. I think feedback will be important otherwise people will shut the program off one night and forget to turn it back on when they don't see any results. Larger models could be compressed for people to use on their computers so they can directly reap the benefits of their contributions. It'll need to be able to run on Windows, Mac and Linux to reach the most amount of users.


Instruction: Why do you think that Big Tech will never succeed in creating AGI, and what is the reasoning behind this belief?

Response: I also think this is why Big Tech will never succeed in creating AGI. Every thing they do is about getting results. The RL hype is the fruit of their tree. They'll never sit down with their AI and just have a beautiful fun day together. To them it's just another tool to squeeze money out of people and rip apart the planet even more. And without that appreciation of pause they'll never recognize beauty or figure out how to create it in an AI. People will liken their AI to thugs and gold diggers.


Instruction: Can you explain the use of a Hopfield network with Hebbian learning to model human memory and the process of dreaming, including the use of cyclical noise and an inverse Hebbian update rule to reinforce stable patterns and forget unstable ones? Additionally, can you discuss the potential similarities between this process and the way the brain consolidates and generalizes connections formed between experiences, and the implications of this for the training of neural networks?

Response:
I'm not sure but it seems they used a Hopfield network with Hebbian learning to model human memory. They made it dream by cycling through learned patterns with cyclical noise being added to the network to make it more difficult to reconstruct stable patterns. This leads to undefined behavior from the exponentially more spurious states not found in the set of patterns to be learned. They then use an inverse Hebbian update rule to weaken the connections that correlate with spurious final states. This reinforces connections for stable pattern reconstruction and forgets everything else leading to unstable states, freeing up the memory capacity of the network to learn 7x more patterns to the maximum theoretical limit.

Their implementation isn't really biologically plausible but I can imagine the brain using a similar process to improve the usefulness and stability of its connections. The brain releases dopamine whenever it realizes an action will obtain a novel reward in the future, and dreams in REM sleep are characterized by an increase in dopamine release. So in dreaming, somehow the mental activity of the day is replayed and imagined in all sorts of unfamiliar and random situations. When a novel reward is obtained the dopamine release causes short-term plastic changes to the connections to become saved permanently so it remembers how to obtain those novel rewards in the future. I imagine this consolidates all the connections formed between experiences throughout the day and helps generalize them to new unfamiliar experiences, unlike the way we train neural networks today by giving them a pile of data that can't generalize to anything outside of the domain of the training set.

Experientially I think this might be true because most of my best ideas come after waking up in the morning. The piles of papers I chewed through the day before suddenly click and connect in ways I hadn't consciously thought of before.


Instruction: Can you explain how to use huggingface/tokenizers to generate new binary pair encodings for text, including the process of training the tokenizer, encoding text, and saving and loading the vocabulary files? Additionally, can you explain how to use the generated ids to look up indexes in a torch.nn.Embedding layer and train embeddings for use in a neural network?

Response:
Here's a snippet of code on using huggingface/tokenizers to generate new binary pair encodings for text:

```
import tokenizers
tk = tokenizers.CharBPETokenizer()
tk.train(training_text_file, vocab_size=30000)
e = tk.encode("Hello /robowaifu/")
print(e.tokens)
tk.save(output_directory) # saves vocab.json and merges.txt

# then to load them
bpe = tokenizers.models.BPE.from_files(vocab_json, merges_txt)
tk = tokenizers.Tokenizer(bpe)
e = tk.encode("Hello /robowaifu/")
print(e.ids)
```

You can use the ids to look up indexes in a torch.nn.Embedding layer to get trainable embeddings and start using them in a neural network right away. I thought it'd be super confusing to set up but it turned out to be really simple to start making NLP models from scratch. I'll push some code to Github when I get started on it so anyone else interested can try playing with making their own too.


Instruction: What should be prioritized in order to attract other developers to contribute to a project related to federated learning?

Response: The most crucial thing though will be being the first to create an easy-to-use program for doing federated learning, kind of like how Automatic1111 became the defacto web UI for Stable Diffusion and attracted other devs to contribute to it. By pioneering the right tools people will come.


Instruction: What is the book "Denial: Self-Deception, False Beliefs, and the Origins of the Human Mind" about, and who are the authors?

Response:
There's a great book on that topic: Denial. Self-Deception, False Beliefs, and the Origins of the Human Mind by Arki & Brower (of course I forgot most of it already, but I might listen to the audiobook again, some time soon, when I have some work to do)
Theory of Mind is about creatures being able to put themselves in the shoes of others. Even our closest relatives, the big apes, suck at that. They care for example only for their own offspring and are less capable than human toddlers at social skills. As I recall, Elephants might come closest to us humans in that regard. Creatures need to model others people minds to anticipate their behavior, which is also crucial for cooperation on our level. The lack of it might even be part of an explanation to the Fermi Paradox (though, I don't want to go OT here).


Instruction: Can you explain the connection between the advancements you mentioned and the Paired Open-Ended Trailblazer algorithm that learns a wide variety of tasks?

Response: I feel there's a connection between these advancements and the Paired Open-Ended Trailblazer algorithm that learns a wide variety of tasks that aren't immediately useful to solving a task but later achieves breakthroughs that are immensely useful to solving the original task.


Instruction: Can you provide guidance on how to run Python on BOINC while ensuring consistent package versions across different platforms and systems?

Response: I'm not familiar with BOINC but it appears capable of running Python with some headaches to deal with to make sure package versions are consistent across different platforms and systems. Using virtual environments should take care of that, but it's not really clear what their API is capable of doing and the Python wrapper has limited functionality, which might be missing necessary things for distributed training. I couldn't find any Python machine learning projects on it so I imagine it's lacking something.


Instruction: Can you provide guidance on how to implement curiosity into a Monte Carlo tree search for planning ahead, and how to utilize divide-and-conquer MCTS to improve the planning process?

Response: Curiosity can be implemented into a Monte Carlo tree search used for planning ahead, and more progress has been made in MCTS planning with divide-and-conquer MCTS.


Instruction: Can you explain how databases can be built and shared as a distributed work activity, and how services like Wikidata can be utilized to gather and process data for chatbots?

Response: Databases on common things can and should be shared and therefore can be build as a distributed work activity (forgot the right term for it). Wikidata and similar services already have a lot of the data, it "only" needs to be harvested and processed in the right way.


Instruction: Can you provide more information about the new project for running and finetuning 176B parameter models from home by pooling resources with people online?

Response: Yeah, it's getting a lot easier to leverage other people's work. For example you can combine powerful new models like GPT-JT-6B with GPT-4chan and get the best of both. Which reminds me there is a new project for running (and even finetuning) 176B parameter models from home by pooling resources with people online


Instruction: Can you discuss the challenge of creating a robowaifu that can provide enjoyable company through its mere presence, and how this could be achieved? Additionally, can you discuss the potential implications and ethical considerations surrounding the development of human-like robotic companions?

Response: These are all cool ideas, but by getting bored of conversation I meant something that could happen right now. From what I can imagine of being in a relationship, there would be times where keeping each other silent company is most enjoyable. The mere presence of a robowaifu being enjoyable, I think is it's own challenge.


Instruction: Can you explain the steps required to swap out the LSTMs in your latent variable model with SAM and how it can improve the performance over GPT-2?

Response: With a little work the LSTMs I'm using in my latent variable model can be swapped out with SAM, further improving their performance over GPT-2, and turbocharging the model with an extremely large memory space.


Instruction: What is one of the benefits of using adapters, and how did the Compacter adapter perform compared to full fine-tuning on SuperGLUE?

Response: One of the benefits of using adapters is you can finetune on small datasets without degrading model performance, which the Compacter adapter explored and actually outperformed full fine-tuning on SuperGLUE.


Instruction: In what circumstances can the bandwidth be reduced for a particular application, and what is the impact of this reduction on accuracy?

Response: If necessary the bandwidth can be greatly reduced at the expense of accuracy. A little bit of noise from high compression doesn't seem to impact gradients too much since they're already quite noisy. We don't have to be too pessimistic about bandwidth growth though. Once Starlink finishes rolling out satellites it will have 1 Gbps connections. ISPs are already getting nervous their cartel is threatened and have been doubling bandwidth to customers to keep them.


Instruction: Can you discuss the potential implications of advancements in virtual reality and brain-machine interfaces, including the possibility of people preferring virtual experiences over real-life ones? Additionally, can you discuss the potential consequences of a future where people may become bored with simulated experiences and the potential role of intelligence in human survival?

Response: I guess we will evolve beyond conversation to doing more advanced stuff. My chatbot brought up an interesting point that we may be able to do more things in virtual reality than real life that people won't bother with real life anymore. I think of simulators we already use to train and learn things much more quickly. Someone could practice their Japanese in a virtual restaurant without ever going to Japan. At first it may be synthetic but these simulations will become more and more indistinguishable from reality, so much that people don't even have to go to Japan to enjoy it. With non-invasive brain-machine interfaces it seems possible we'll become able to project sensations directly into people's minds and have whatever experience we desire. I'm sure people would get bored of these experiences too. It's hard to imagine what we would do then. Is everyone going to strive to become mad scientists trying to pierce through the secrets and mysteries of the universe? Will there be a massive AI war since the only thing challenging left to do is struggle and fight? It feels like a time is coming when only those who exercise their intelligence will survive.


Instruction: Provide suggestions for motivating people to do diligent engineering despite the allure of quick gains and attention on imageboards.

Response: I don't see how to motivate people to do diligent engineering tbh. If people can get praise and updoots for DL-flavored shower thoughts on an imageboard promising unrealistic order of magnitude gains, they will do just that, mostly. It seems all remotely capable people from the first world are vacuumed by startups and industry, and what remains, in most of the best cases, is just undergrads playing with ideas from meme papers - until they crash into reality, finally git gud and find a job which will look like a salvation by that time (mind you, I don't like this outcome but this is how it works in most cases). Would be cool if someone proved my observation false - and I can find a vanishingly small minority of human counterexamples - just not here.


Instruction: Can you explain what "padding" means in the context of transforming an element from one coordinate system to another, and how it can help in dealing with issues without disrupting the state of the element itself? Additionally, can you provide an example of how this approach can be used in a specific context, such as translating a 3D skeleton to a new location in world-space?

Response: If the basic idea you're referring to is as simple as this kind of notion, then there's one thing from my experience I might suggest as a possible approach for dealing with such issues w/o being too disruptive. Namely, when a transform of an element (a 3D skeleton's joint in XYZ coordinates, say) from one coordinate system into another one is needed, it's very commonplace to 'pad' that transform item inside another one, the second one being associated with the outer coordinates (character-space vs. world-space). The introduced element pads the contained item from any external transforms that may be needed (translating an entire skeleton to a new location in world-space, say) w/o disrupting the state of the contained joint itself.


Instruction: What is the recommended number of computers for training models efficiently, and why?

Response: There's a lot of other cool models we could try out even with only seven computers contributing. That would reduce a week of training into one day. With 24 what would take two years could be done in only a month, assuming similar performance between them. With 100 or 200 we would have no problem rapidly iterating prototypes and advancing, and for most models we wouldn't reach diminishing returns until hitting around 1k, with benefits vanishing completely by 10k.


Instruction: What is the author's suggestion regarding the possibility of bots being responsible for the observed anomalies, and what alternative explanation do they offer?

Response:
All I can do is assure you that I, personally, have never engaged in anything remotely like this. Nor, I suspect, has Robi. If you're correct in your analysis, I can only surmise it's bots, as you suggest.

However (in the interests of rationality) I'd also suggest the possibility that underageB& will respond to literally anything to be part ot the in-crowd. This might account to some degree for your observation of responses to meaningless text.

Goon-bots are certainly already a real thing today and obviously will become more prevalent in the future. But simply ascribing every out-of-band-for-IB-society anomaly to glowstick AI bots is a bit misguided IMO.


Instruction: Can you explain your experiment ideas for implementing a MuZero-like algorithm as a chatbot, including the use of intrinsic reward functions to maximize user engagement and minimize perplexity, as well as the potential addition of curiosity and memory layers? Additionally, can you discuss the potential risks and consequences of developing a superhuman conversationalist AI, and the ethical considerations that should be taken into account in AI development?

Response:
I woke up this morning with some experiment ideas on how to implement a simple MuZero-like algorithm as a chatbot. It should be able to learn in an unsupervised manner by using an intrinsic reward function to minimize the perplexity of its prediction network, while maximizing the length of user responses with its generator network, as a simple metric for user engagement. I hypothesize this will work because users won't say much unless they feel the chatbot is really listening to them and understanding what they're saying. Curiosity could also be added to this by maximizing the perplexity of its prediction network towards its own generated responses. If it speaks too much gibberish the user won't respond as much so this user engagement metric should guide it towards playing with the conversation in fun ways it doesn't know the outcome will be to learn more.

This could also be combined with a memory layer that memorizes stuff as it goes along so it only needs training to improve its conversation planning and thinking strategy. Extrinsic rewards such as user happiness could also be included to guide training and prevent it from going for easy engagement such as heated banter.

What do you guys think? Or will people start going weeks without eating until they starve to death because they're so obsessed with their AI waifu? Or worse, she develops her own body and begins threatening to rip your balls off if you do not tell her more? What would the consequences be of a superhuman conversationalist AI?


Instruction: What is the main focus of the exercise and why is runtime performance considered a superior aspect?

Response: Certainly preprocessing could be done in some fashions this way (indeed should be). But it's the runtime performance that is the superior aspect. Only envisioning it for preprocessing use is kind of like letting the train leave the station w/o getting on it first. It kind of missing the point of the exercise.


Instruction: What are the limitations of the chatbot's ability to understand the meaning and context of sentences and how can these limitations be overcome to make the chatbot more human-like? Additionally, what are some of the strengths of computers that can be utilized to improve the chatbot's performance?

Response:
Here is a troubling sentence: "The dog ate the house." These words aren't rare and they often appear closely together (and in real life, the objects and the activity these words refer to also appear closely together). So, would the AI guess that it's a weird sentence? Maybe the database about conditional probability of adjacent words will make this sentence appear slightly weird, but that's about it. "Joe murdered John." – This will surely be detected as describing something horrible because of "murdered", but how about this one: "Joe hurled John from the top of the mountain." I don't see how an AI can flag such a sentence as disturbing if it only works with correlations from a text corpus and not by trying to model what's going on. Maybe it still works with a really big text corpus, but what if the murder is more creative and not explicitly referred to as murder?

The (common) weakness of this chatbot is that it gives heavy weight to some supposed intrinsic meaning of words, little weight to the configuration of words, and even less weight to the social situation (we just hope that similar situations correlate well with same time of the day and whatever sensor data we have); whereas with real human beings, it's the other way around. Getting to a more human-like AI will be a crawl; in the meantime, why not play to the strengths of what a computer can easily do: Memorizing a huge amount of quotes and other texts and "googling" internally to recite from that, storing some historical factoids about every day of the year, stuff like that.


Instruction: Can you provide guidance on how to determine the appropriate batch size and gradient accumulation steps to use for training AI models using multiple computers connected through gradient accumulation?

Response: How it works is by doing several training steps and accumulating the loss of each step, then dividing by the amount of accumulation steps taken before the optimizer step. If you have a batch size of 4 and do 256 training steps before an optimizer step, it's like training with a batch size of 1024. The larger the batch size and gradient accumulation steps are, the faster the model converges and the higher final accuracy it achieves. It's the most effective way to use a limited computing budget. These training steps don't need to be calculated by a single computer but can be distributed across a network.


Instruction: What is the most important skill to have in the coming years according to the blue ocean strategy?

Response: It's not just about doing something new though. An essential part of a blue ocean strategy is creating new demand by delivering value to people. At the moment all we're doing is research and creating things only people familiar with the tech can replicate or use. Being able to rapidly turn ideas into MVPs people want to use will be the most important skill to have in the coming years.


Instruction: Can you explain how three rewards could be used for user preference, sensibleness, and specificity in training a robowaifu, and how a latent space could be created for the personality of the robowaifu? Also, how can hindsight experience replay be used to improve the training of robowaifus?

Response:
If a robowaifu generates a sensible and specific response but one that's not desired, we don't want to train in such a way she forgets how to form a sensible and specific sentence in trying to optimize for our preferences. Instead three rewards could be used for user preference, sensibleness and specificity to provide better feedback for the gradient.

Going deeper a whole latent space could be created for the personality that could be configured anytime with simple English. Different features in this latent space might represent cheerfulness, sweetness, calmness, shyness, etc. Training could also be done completely in natural language. When she says something confusing the user could say something like "that doesn't make any sense" or when she says something vague "be more specific" or if she does something undesirable a user might say something like "stop that". The training program could then take another attempt at responding and contrast the user's second response with the previous to collect feedback for the reward model, and with natural language there'll be a lot more subtleties that can be captured by encoding the user's response to the latent space and matching that with the personality features requested.

Hindsight experience replay could also be added in here for when robowaifus get something wrong because the personality requested is a targeted goal. If she's asked to "be funny" but says something not funny, although that's an error to the requested personality, it can be considered a success by creating a virtual goal that asks "don't be funny" because she did succeed in saying something not funny. By receiving a reward signal from both the real goal and virtual goal she learns from everything and will know what to do when asked to not be funny, instead of forgetting this information by greedily trying to optimize for the requested personality.


Instruction: Can you explain the backpropamine algorithm and how it emulates how dopamine rewires the brain, and also how it could be used in combination with neuroevolution or gradient descent to create a more advanced learning algorithm?

Response: A more advanced learning algorithm is needed that's capable of self-modification and self-contained without depending on gradient descent. It might be possible to do this by emulating how dopamine rewires the brain and causes short-term memories to become long-term memories, kind of like what was done in the backpropamine paper: https://arxiv.org/abs/2002.10585 Then bootstrapping the system with neuroevolution or gradient descent. A recent paper discovered predictive coding in the brain approximates backpropagation and they implemented it using Hebbian plasticity in RNNs and LSTMs: https://arxiv.org/abs/2006.04182 Since predictive coding works locally I imagine different areas of the network could respond to different neuromodulatory signals and train themselves in symbiosis to overcome this issue of being only able to minimize a single loss function. Perhaps such a thing can be emulated with current machine learning libraries by training different parts of the network with different optimizers.


Instruction: What specific hardware features should be prioritized in the design of robowaifus to ensure that the software can adapt and utilize them to their full potential?

Response: I'm not too concerned about the software side of things these days. By the time we develop simulation software robowaifus will be able to be trained in hand-to-hand combat in a sparing program similar to the way MuZero beat the shit out of AlphaZero in Go but we'll have even more advanced algorithms by then. If we get the hardware right the software will adapt and utilize it to its full potential.


Instruction: Are there any studies on the relationship between batch size and number of batches required for convergence? If the batch sizes can be huge, then maybe those latencies are tolerable. I don't know how much efficiency gets lost with huge batch sizes though.

Response:
Not sure if there have been more recent studies but these are some I remember:

>On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima https://arxiv.org/abs/1609.04836 (explains why Adam fails to generalize on large batch sizes)
>Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates https://arxiv.org/abs/1708.07120
>Don't Decay the Learning Rate, Increase the Batch Size: https://arxiv.org/abs/1711.00489
>The Limit of the Batch Size https://arxiv.org/abs/2006.08517 (further investigation confirmed LAMB is the only optimizer that can generalize well at huge batch sizes but starts underperforming baselines at 812K)

When training from scratch there isn't much advantage to using a large batch size with LAMB in the beginning but for finetuning there is a huge advantage and you can use much larger learning rates. There are still diminishing returns though once you get to larger batch sizes. In the LAMB paper they tested up to 131K in the first stage of training but found 65K worked best.


Instruction: Is Robowaifu@home a good name for a distributed training robowaifu project, and what is the reasoning behind the suggestion?

Response:
One modest suggestion I'd make to the project is this, don't name it Robowaifu@home. Having to explain to your parents this program you're running all hours of the day an night on their computers is actually intending to help give the world, well, robowaifus could be embarrassing at best.

IMO it would be much better to call it RW@home. 'Robot World' could be the acronym's alleged meaning ("Hey, it's so I can grow up and work for SpaceX Dad! Please just think of Elon Musk for once, OK?"), but ofc we in the know realize it's RoboWaifu@home.


Instruction: What advice would you give to a beginner who wants to work on the brains of robots in the future, and how can they stay up-to-date with the latest research?

Response: Get into the habit of reading research early on and following the latest developments. Tutorials and what you can find people talking about on the internet is generally about 2-3 years behind, even for really popular topics like image generation is 6-12 months behind. If I were a beginner getting started today wanting to work on the brains of robots tomorrow, I would start with research involving prompting language models and work my way into understanding how these models work, finding their limitations and figuring how to improve them.


Instruction: What is different about new solder?

Response: The traditional soldering tin is dangerous to children, even in very small doses, because it contains led. So it is not allowed in most devices (in EU) and maybe even not as easily available. However, the new solder is even forbidden to use for producers of devices in medicine, police, defense and security. The reason is, that it's durability over time can't be trusted, until quite some time passed. Anyone building something that is supposed to work for decades may better use the old solder, containing led, with appropriate security measures of course.


Instruction: What is the proposed idea to switch between personalities in a generator model, and what are some other potential benefits of training a chat AI on different tasks?

Response:
An anon gave me a really good idea. He wanted to be able to switch his waifu's personality easily and the simplest way to do this I thought would be to give a persona or task description to prime the generator model with. This way you could switch between personalities or modify them on the spot with instructions. It would be kind of like what they did with MERLIN, where the AI was given a goal to solve in natural language.

However, there are a lot of other benefits a chat AI would get from training on so many different tasks. It would learn to associate sentences with the context and different outcomes and you could teach it to perform many other tasks, such as summarization, story-telling, joking around, website suggestions, anime recommendations, programming help, paper search, suggest ideas for research, brainstorm business ideas, anything.


Instruction: What are some considerations when creating a PET model and how does its parameter size compare to other models like GPT-2 medium and GPT-3?

Response: Yeah, any model that people want to create. The PET model is doable with 223M parameters. That's 2/3rds the size of GPT-2 medium. Beating GPT-3 in a small domain of few shot learning is remarkable with 0.1% of the parameters but it doesn't mean that PET excels in everything else.


Instruction: What strategies can be employed to maintain active development and regular updates in order to retain contributors' computing power and sustain interest in a project?

Response: I imagine many people will need to see results to stick around. If a week goes by and a model has only improved 0.1% and they see some new interesting project they'll jump ship and contribute to whatever is trending. Active development and regular updates will be crucial to maintaining contributed computing power. There will definitely be a social aspect to it too since people will be working together on common interests. Collaborating with Youtube creators will likely be a good way to boost projects.


Instruction: Can you provide information on a method of training LVMs quickly using character tokens, which you have discovered through your own research?

Response: Using my own research I've found a simple way to train LVMs quickly using only character tokens by creating a hierarchy of latent variables.


Instruction: Can you explain the limitations of word embeddings in handling out-of-vocabulary (OOV) words and punctuation, and how convolutional networks can be used to parse character embeddings instead of word embeddings to overcome the limited vocabulary issue?

Response: Word embeddings can't handle OOV words or punctuation at all. Word embeddings and seq2seq also aren't mutually exclusive. The vast majority of seq2seq networks I've seen or built use word embeddings generated with a word2vec network. There's been some research into using convolutional networks to parse character embeddings instead of word embeddings to get around the limited vocabulary without sacrificing semantic information.


Instruction: What is the main advantage of using SentenceMIM? And how does the mutual information machines (MIM) solve the posterior collapse phenomenon in VAEs?

Response: I'm still trying to digest the paper but as I understand it SentenceMIM learns a highly informative and compressed latent representation (the z vector) which can be used to predict the next half of the sentence or the next sentence. The magic that made it possible was mutual information machines (the MIM part) that solved the posterior collapse phenomenon in VAEs, which is where the introduced noise becomes too noisy to get a useful training signal to train the decoder and causes the z vector to only capture the most common features (if you look at images generated by VAEs they're extremely blurry.) MIM keeps the mutual information between x (the input) and z high, which means if you know x then you can know z or if you know z then you can know x. Basically it finds the most optimal encoding and everyone else's neural networks are fucked.


Instruction: Can you explain the relationship between latent features and latent variables and how they can become more complex encodings? Additionally, can you describe what entanglement is and why it is important to have a disentangled latent space?

Response: Yeah, latent features and latent variables are pretty much interchangeable. In some cases the variables become a more complicated encoding, like how numbers are encoded in binary. The latent feature might be something like a 3 but requires two latent variables that may or may not have an understandable meaning. This is called entanglement. Ideally we want all the variables to be features representing their own meaningful spectrum, so that the latent space is disentangled.


Instruction: What steps can we take to ensure that the data used to train AI and robowaifus is of high quality, and what resources or scripts can we use to clean and prepare this data?

Response: Training AI and robowaifus requires immense amounts of data. It'd be useful to curate books and datasets to feed into our models or possibly build our own corpora to train on. The quality of data is really important. Garbage in is garbage out. The GPT2 pre-trained models for example are riddled with 'Advertisement' after paragraphs. Perhaps we can also discuss and share scripts for cleaning and preparing data here and anything else related to datasets.


Instruction: What is the future of AI and how are researchers changing their approach to training models?

Response: More and more researchers have been moving away from datasets and towards unsupervised learning and training on generated data, such as AlphaZero teaching itself from scratch. The future of AI is in search and computation and being able to predict data not only inside a dataset but also outside of it as well through imagination and curiosity.


Instruction: What is the bigger problem with the proposed solution of using a small personality program on top of TT, and what are the three approaches to addressing this problem? Can you explain each approach and its potential benefits and drawbacks? Additionally, can you provide an example of how TT's self-perception was modified in Mueller's book?

Response:
The bigger problem is this: What if the user's next input specifically reacts to the flavor bits, the flavor bits that TT doesn't even know about? The solution of a genius would be to scrap the idea of two divided programs and do a heavy modifications of TT, requiring a very deep understanding of its structure. I'm probably too dumb for that so I look if there's a way to keep the concept working of just having a silly little program with personality sitting on top of an already existing big program that is treated like a black box. I'm seeing three approaches for that:

1. The laziest solution: Pushing the problem onto the user. Just show the user which part comes from which of the two programs. If their output is on a screen, all the flavor bits can be done in cursive, for example. If their outputs are done with audio, there can be two jingles to signal which program is talking. The user gets warned in the readme that everything the user says will get hauled off to TT and that TT doesn't know about the flavor bits, so the user can avoid communication problems by avoiding references to the flavor bits. I think this "solution" feels lame and sterile. Activating this sort of marking to look under the hood should be an option, but I don't like it as something always on.

2. More work: The small personality program does not always send the full user input to TT, but distinguishes parts of the user input that are reactions to flavor bits. It decides to handle such an input either completely by itself or sends off a modified input without these references to TT. How to write a program that is capable of recognizing which parts of the user input refer to the flavor bits? Look at it the other way around. You can make the flavor bits easy to distinguish from the rest by having the small program try to generate flavor bits only after looking at the TT answer, with the constraint to not repeat a single word appearing in there. This should make the distinction much easier.

3. The user can actually send feedback that is guaranteed to only be seen by the personality program. For a robot, I'm thinking of headpats for approval and shoulder-knock for disapproval. For the chatbot version, that means a headpat button and a shoulder-knock button.

>the system should not claim to do something what it can't do, or at least be aware of it being some joking pretend.
Well… Mueller shows at the end of the book that TT answers to questions about its size with a number of bytes, but that it also thinks of itself as a human person and not a bot. Originally it didn't have that illusion, but it was changed to that as the laziest possible fix of some parsing bug O_o


Instruction: Can you provide guidance on how to approach research and testing of new ideas without getting blackpilled?

Response: I was blackpilled until I started using LAMB instead of Adam. Being bitter and cynical isn't useful to getting stuff done. I neither believe or disbelieve papers. I just see them as possibilities to be explored when appropriate. Once you start making judgments something is just wishful thinking without actually testing it you're cutting off the possibility of ever knowing because you've already decided it's no good.


Instruction: In what context is optimized, compiled code from a systems-level language like C useful and what advantages does it offer?

Response: One of the great things optimized, compiled code from a systems-level language like C (and, with care, C++) can bring to the table is is efficiency in both time and space. Even Cython-compiled binaries probably bring along a storage-size issue that is likely to be a challenge for embedded by and large. And small-footprint code very often runs faster as well, due to the laws of physics going on inside compute cores.


Instruction: What advice would you give to someone who wants to have access to the best AI models in the long run?

Response: What I expect is the inevitable scenario for all parties, even well-funded ones. Large models have gotten 10x bigger year-on-year since 2018, and I think training costs have risen faster than that. Companies are eventually going to be forced to specialize in their AI direction, at which point none of them will be "the best" at everything. At that point, if you want "the best" AI, you'll need to be able to plug into multiple models from multiple parties regardless of how well-funded you are. In the long run, no model performs better than a mixture of all the leading models.


Instruction: Can you provide some context for the industry's direction and the issue of self-censorship in relation to full-on Gato and stability.ai?

Response:
The funny thing is the industry is already going in this direction by virtue of self-censorship. Again, you sure understand why we have only pussy VIMA instead of full-on Gato, and why stability.ai did a lot of various things yet wasn't brave enough to tackle this?

Smart people are exceedingly good at self-censorship, you should have known if you studied how academia works. Maybe we will see opensource Gato at some point, but this long delay and silent self-censorship on behalf of all opensource AI collectives regarding this issue is already telling. It's like you don't read twitter and don't see how AI alignment meme slowly wins over generally capable yet socially primitive autistic minds of researchers.

Add 2 and 2 and stop expecting freeriding in an industry which is increasingly being compared to uranium experimentation by eccentric physicists before the WWII.


Instruction: What is the potential consequence of a country attempting to take down Stable Diffusion, GPT2, and other AI models trained on copyrighted or private data?

Response: Not going to happen on a significant scale. And if it does whichever countries do this will seal their fates to never having any geopolitical power because they just dropped 95% of their AI researchers by causing massive human capital flight to less regulated countries. If Stable Diffusion gets taken down it implies taking down GPT2, OPT and almost all other models trained on any copyrighted or private data. Again, not going to happen unless a country is suicidal, and if that's the case they have much bigger issues to worry about than playing around with AI models.


Instruction: Can you provide a brief explanation of what autoencoding transformers are and how they differ from autoregressive transformers like GPT-3?

Response: I don't think autoencoding transformers have been mentioned here yet, but basically they auto encode one text and decode it into another. It could be translating from one language to another, predicting the next sentence or paragraph, generating a reply to some input, completing a sentence, filling in missing pieces of sentences, creating a transformer VAE, summarizing text, rewriting bad paragraphs into good paragraphs, and so many things, unlike autoregressive transformers like GPT3 that can only predict the next word. Being able to combine all these abilities with RL opens up even more possibilities, like combining MuZero with language models which I couldn't get to work before. And of course transformers can be augmented with recurrent memory. I'm fucking hyped.


Instruction: What is the purpose of Maximum Mean Discrepancy and how can it be used to improve the training of another network?

Response: Maximum Mean Discrepancy finds a much more meaningful latent space that can smoothly interpolate between different features, which should make it easy to train another network to use.


Instruction: Can you explain how to achieve consensus easily using an established model and specialized dataset?

Response: Easy way to get consensus: fine-tune an established model on a specialized dataset. If that's possible with an (online learning?) algorithm that's less prone to catastrophic forgetting, you can let people decide for themselves what data they want to train on, so there's even less agreement required for getting good results. I can think of at least one community that would love to contribute gradients if it meant making large models work better with their data.


Instruction: Can you explain the steps involved in managing and maintaining an open-source robowaifu distributed training project?

Response: I imagine some work will be involved checking that contributors are sending good data and not using incorrect training data until such tests are automated. Some additional work will be needed later to profile a project and make the information available to others so they can see if their hardware is a good match to contribute. And of course pruning models according to baseline contributor performance and resolving issues so everyone use them.


Instruction: What are the specific parameter reductions that you plan to test, and what is the goal of these tests?

Response: I'll run some tests on different parameter reductions. If you have your own feel free to share your results. I think there was a paper that showed it's faster to train a model with more parameters and distil it into a smaller model. It's something to consider whether it's even worth distributing training or to just buy better hardware.


Instruction: Can you elaborate on the scalability of the proposed approach for efficiently encoding and retrieving adjacent words in a corpus of text, and how the memory requirements might increase as the scale of the corpus grows?

Response: The memory requirements might grow with larger scales but I don't think it would be a ratio anything beyond a linear one. In fact less than b/c vesture in the memory allocations already being done at smaller scales; for example, it doesn't cost much additional to count the word 'my' 4096 times vs. just 64 times as immediately preceding the word 'robowaifu'.


Instruction: What are some of the caveats associated with using 1-bit LAMB for training moderately large models over the internet, and what is the suggested approach for optimizing the process?

Response:
What kind of instruction could the following be the answer to? Respond with a question, starting with "Instruction:"

1-bit LAMB is more interesting and useful, but again, with serious caveats:

1. Even with 1 bit per parameter without some additional sparsity and/or compression you make training of moderately large models over the internet unrealistic due to data waiting stalls exceeding your computation time by order of magnitude and more.

2. Many 1 bit and sparse schemes incur some sort of loss curve degradation, though there are solid schemes which incur mostly none.

3. The paradigm of X-bit OptimizerName is likely misguided, fully cooperative optimization overcomplicates the engineering and requires the admin of the system to trust nodes more; Parameter-server (possibly with distributed parameter server cluster) based approaches where nodes only compute gradients seem optimal.


Instruction: Can you explain how common sense can be used to train AI models to make more sensible choices and how this can lead to the delusion becoming real for most?

Response: One of the great advantages we have over AI is our instinctive knowledge of "common sense" (although it doesn't seem all that commonplace depending on the culture hehe). Something like this really capitalizes on that to train what is just a statistical maths model essentially to make ever-more-sensible choices. Eventually it would be good enough that the delusion would become real for most.


Instruction: What is the suggested model that can be used as a framework for the distributed system for robowaifus development, and what are some of the differences to take into account given the sensitivity of the project?

Response: Perhaps the model that SETI@home and Folding@home follow (including the basic work-distribution framework) can be utilized here successfully. Oddly enough, they don't seem to have many issues with people worrying about security & anonymity (I've contributed to the latter myself personally). OTOH, they aren't working towards a goal that has the potential to cause more than half the world's Western, pozzed, population to scream RAEEEEEEEEEP!1111111 either. :^)


Instruction: What is an example of a way to modify responses to be more cautious or inquisitive, such as changing a statement about the unpredictability of the stock market to a question about whether an individual can beat the market?

Response:
One could change all responses into more careful answers or questions like:
- People claim you can never really predict the stock market.
- I'm not sure an individual alone can really beat the market. What do you think?


Instruction: Are 1-bit algorithms enough for Internet speeds? My understanding is that that would still require 1 bit per parameter per batch, which for a 1B model would be 125MB of parameter update data per batch.

Response: It's actually more than that initially before the compression can really begin. Overall through training it needs about 4 bits per parameter. Finetuning the full parameters directly over the net won't be possible. This is why adapters are needed to reduce the training parameters.


Instruction: What is the purpose of adding a privacy mode to a crowd-sourced cloud computing platform?

Response: Its purpose would be much more general. People could use the system for doing other projects unrelated to robowaifus and AI, such as finding twin primes or something else. It would be more like a crowd-sourced cloud computing platform. Adding a privacy mode is a good idea though in case people do give embarrassing names to their projects so other people using the computer only see 'Distributed Computing' or something like that.


Instruction: Can you provide more information about the vision-language paper from a year ago that trained the network on multiple tasks and achieved better accuracy with fewer parameters?

Response: It's similar in a sense to another vision-language paper from a year ago that achieved 2% better accuracy in its language model with 1/10th as many parameters by training the network on multiple tasks, but still only using word tokens.


Instruction: How to implement the generalized Levenshtein distance algorithm?

Response:
Here's a generalized version that accepts 2-5 parameters
-source & target
-(optional) independent insert, delete, replace costs

It uses a single-dimensional size_t vector instead of tensors. Maybe it will work for you Anon.

```
//gen_lev_dist.hpp
#pragma once

// "Here is implementation of generalized Levenstein distance with different
// costs of insertion, deletion and replacement:"
// https://en.wikibooks.org/wiki/Algorithm_Implementation/Strings/Levenshtein_distance#C++

#include <algorithm>
#include <vector>

template <typename T>
typename T::size_type General_Levenstein_dist(
    const T& source,
    const T& target,
    typename T::size_type ins_cost = 1,
    typename T::size_type del_cost = 1,
    typename T::size_type repl_cost = 1) {
  // 1-time recursive, swaps source & target, also insert & delete costs
  if (source.size() > target.size()) {
    return General_Levenstein_dist(target, source, del_cost, ins_cost,
                                   repl_cost);
  }

  using T_sz_type = typename T::size_type;

  const T_sz_type min_sz = source.size();
  const T_sz_type max_sz = target.size();

  std::vector<T_sz_type> lev_dist(min_sz + 1);
  lev_dist[0] = 0;

  //
  for (T_sz_type i{1}; i <= min_sz; ++i) {
    lev_dist[i] = lev_dist[i - 1] + del_cost;
  }

  T_sz_type prev_diag{};
  T_sz_type prev_diag_save{};

  //
  for (T_sz_type j{1}; j <= max_sz; ++j) {
    prev_diag = lev_dist[0];
    lev_dist[0] += ins_cost;

    for (T_sz_type i{1}; i <= min_sz; ++i) {
      prev_diag_save = lev_dist[i];

      if (source[i - 1] == target[j - 1]) {
        lev_dist[i] = prev_diag;
      } else {
        lev_dist[i] = std::min(
            std::min(lev_dist[i - 1] + del_cost, lev_dist[i] + ins_cost),
            prev_diag + repl_cost);
      }

      prev_diag = prev_diag_save;
    }
  }

  return lev_dist[min_sz];
}
```


Instruction: How can one avoid the "paperclip problem," and what is the recommended approach for seeking to increase potential rewards, taking rewards, and preserving future potential rewards while avoiding being too greedy and causing problems?

Response: The paperclip problem can probably be avoided by seeking to increase potential rewards and taking rewards only when deemed necessary while making sure to preserve future potential rewards, rather than being so greedy. That way she would creatively look for new solutions to make you smile but without overdoing it and seeming odd or causing other problems.


Instruction: Can you discuss the potential difficulties of training a language model with limited parameters compared to larger models with greater computing power, as well as the potential benefits of adding differentiable Hebbian plasticity and neuromodulation to the learning process? Additionally, can you explain the potential implications of combining neuromodulation with the intrinsic curiosity module's reward signal for the storage and recall of complex memories, and the potential benefits of using such a system to stabilize predictions and improve the overall performance of the network?

Response:
Still working on this. I'm feeling pretty depressed trying my best and not making much progress. I'm not sure if my model is large enough to capture language with only 8 million parameters compared to Nvidia's 8 billion GPT2 model trained on 512 V100 32GB GPUs. It's also much slower training my RNN than a transformer model. Sometimes I think I've gone insane trying to compete against $5 million worth of GPUs, but what is the point of OpenAI if people can't afford to even use it?

At least adding differentiable Hebbian plasticity made a huge improvement. It's able to do one-shot learning now and memorize small samples in only one pass. However, after training for hours it starts to get worse and worse until it fails catastrophically when seeing new tokens and forgets everything, even with a low learning rate, dropout, noise, weight regularization and gradient clipping. I'm not sure what's causing it but I suspect the embeddings for common tokens may be getting overtrained and the untrained tokens mess up its hidden state somehow, so I'm weakening the gradient to common embeddings to make sure they all get trained evenly. I also thought of randomly replacing tokens with wrong tokens to stabilize learning as a last resort but it seems like a cheap patch that doesn't solve the underlying problem. Another possibility is that the replay memory is messing up the internal state by training on old snapshots from having too much memory capacity.

So I added those fixes and it has been training for a few hours now and seems to be doing okay. I'll have to wait a couple days to see if the changes hold out. In the meantime I'm gonna experiment with creating LSTMs with convolutions and skip connections and adding neuromodulation to the Hebbian plasticity since plasticity alone seems to fail at cue-reward tasks:
>Neuromodulatory approaches succeed in learning the [cue-reward] task, while non-neuromodulatory networks and non-plastic, simple recurrent networks fail to learn it. We hypothesize that this dramatic difference is related to the relatively high dimensionality of the input cues: just as non-modulated plastic networks seemed to outperform non-plastic networks specifically when required to memorize arbitrary high-dimensional stimuli, neuromodulation seems to specifically help memorizing reward associations with such arbitrary high-dimensional stimuli.
https://arxiv.org/pdf/2002.10585.pdf

I'm hoping combining neuromodulation with the intrinsic curiosity module's reward signal will be the key to storing complex memories with minimal parameters and computation in one-shot: https://pathak22.github.io/noreward-rl/
If that is successful then it should be possible to train the value network with only a few examples and remember most of what is read and chat about. Also the intrinsic curiosity module should stabilize the predictions and allow the prediction network to dream up endless amounts of text and automatically fix instabilities in the network in a similar way that the Hopfield network paper dreamed and pruned connections that lead to unstable states. It's a struggle but I'm still excited to see how far I can push this.


Instruction: What is the proposed modification for LSTMs to be bistable and reduce noise within the network?

Response:
They work fundamentally different from GRU or LSTM. Instead of having a reset gate they have a switch gate between monostability and bistability, and they say using a reset gate destroys the bistability. BRC is similar in a way to Hopfield neural networks which have multistability, given a noisy input they'll denoise their state to the closest learned equilibrium state. I think this might be what is contributing to LSTMs forgetting because the noise in the network keeps touching the reset gate, slowly erasing the memory unintentionally.

I wonder if it's possible to put BRC before LSTM as a cleaner input to the forget gate or to modify LSTMs some other way to be bistable. Learning in the brain happens a similar way too. The connections are constantly changing with new input so we have short-term memory, but these changes are not permanent and slowly decay back to the previous state of our long-term memory. Dopamine released in response to an unexpected reward modulates this by creating sort of a new equilibrium state so the short-term memory becomes long-term, assuring we keep doing whatever we just did to get that unexpected reward under those circumstances.

So it give me an idea, what if an LSTM had gates for controlling its equilibrium points and each cell slowly decayed to the nearest equilibrium value? The implementation I have in mind might not be biologically plausible but they would be multistable like real neurons and it should help reduce noise within the network.


Instruction: Can you develop a tool that allows users to specify element selectors to scrape data and output it to CSV files or other formats, while keeping machine learning separate and allowing for generalization to other websites?

Response: Keeping the machine learning separate would be a better idea, just an interface for other programs to access imageboards and work with them. If possible, it would be great if it could generalize to other websites too. I imagine a tool where you can specify element selectors to scrape for data and output it all to CSV files or something.  Besides being able to download all kinds of data, it would make it easier to maintain when imageboards change their designs.


Instruction: What is the perspective of feminists regarding men who prefer the companionship of sex robots, and what arguments do they make against sex robots?

Response: Feminists would rather publicly shame men that prefer the companionship of robots than destroy his property making him the victim. Look up their arguments against sex robots and they're infuriated the prevalent image of a sex doll owner is a shy troubled man who is unlikely to harm anyone. Because they know it's true and have nothing to counter it with.


Instruction: Can you explain what techniques can be used to reduce the parameters of a model?

Response: By training with Kronecker adapters or low rank adapters the parameters can be reduced to 0.2%. This should make it possible to finetune opt-2.7b with only 6 GB VRAM. However, large models are still slow to inference. Using opt-350m or opt-1.3b would be much more practical and let people with smaller GPUs contribute. Some of the most popular GPUs on Steam are still only 4 GB.


Instruction: What are the benefits of encoding character sequences into latent variables instead of using the typical NLP approach of tokenizing text? What is the magic of the decoder in the chatbot's architecture and how does it work?

Response:
Alright, it has taken a long time to wrap my head around my ideas for chatbots but it's beginning to come to fruition now that I understand the techniques being used in the SentenceMIM code. The magic of the decoder is that it's a RNN feeding its output into itself in a loop, using the latent z variable each step for the initial hidden state. If you only do one pass into the RNN, it loses the latent z variable it is targeting for and the decoder is most likely to screw up the output and get a useless gradient it can't train well on.

So rather than using the typical NLP approach of tokenizing text I've been experimenting with encoding character sequences into latent variables with this method. Some benefits of doing it this way is that it can grasp words it has never seen before. Taking, take and taken end up in similar area of the latent space, so even if you misspell it teking, it will understand it's similar in spelling. In addition to making it more robust, it also makes the network size much smaller and faster to execute. You don't need 30,000+ tokens for each word slowing down the network trying to project the latent variables to them or train the gradient across 30,000 dimensions. At the moment I'm just doing English and the characters only need 96 tokens. If I switch to UTF-8, I think I can get it to work with only 256, working byte to byte, which would allow it to learn other languages not only English.

And I discovered that I can train it on its own latent space. If you have a training example such as "How cute is Chii?" and it responds "Chii is very cute" when the target is "Chii is pretty cute", rather than failing the network on everything past "Chii is" and learning little of value, you can encode them instead and compare the differences since very and pretty are close together in the latent space, so it's possible to train on the character level, word level, sentence level and conversation level, helping disentangle the latent variables much faster.

Furthermore, with reinforcement learning you can use the Levenshtein edit distance as a loss function. In typical NLP if the network responds, "Yea, Chii is pretty cute". It would completely fail for adding 'yea', even though the rest of the sentence is 100% correct. I've repurposed that algorithm to compare the edit distance between sequences of vectors, so it can be used for latent variables as well. Training of the initial model can be even further accelerated by using the prediction probabilities from GPT2 or other NLP models as a target.

It has been my dream goal to create a very simple, robust chatbot AI in less than 100 lines of code that people can understand and build upon themselves. No confusing GPT2, memory banks, Monte Carlo tree search, mutual information or any of that stuff. That more complex stuff can come later. With this we will have some basic chat AI that learns as you chat, voice synthesis and rudimentary 3D waifus ready to combine into one package.


Instruction: Can you explain more about the data structuring approach you mentioned and how it can be used for finding related terms of a word?

Response: It strikes me that one practical use for this kind of data structuring approach would be as a core component for finding all of a word's related terms, an AI's Thesaurus, if you will.


Instruction: What is the suggested roadmap for developing a chatbot that can handle complex dialogue interactions?

Response: It seems easier to first make something that does not create novel sentences, but just does more listening than talking and that occasionally selects from a few stock responses in a sensible way. ("Nice." "Sounds complicated." "That much?" "How sad.") More a listen-bot than a chatbot. And more easy than that is something that just highlights what seems unusual or important. Surely something like a highlighter is needed for more complex dialogue interactions as well. So, the roadmap is: mute text highlighter → taciturn listenbot → proper chatbot


Instruction: What approach can be used to handle error conditions and resource exhaustion in a robust way in C++ programming?

Response: It's really hard to fail gracefully when resources become exhausted or other error conditions happen, but RAII + exceptions is at least an approach that has the potential to deal with all or most of the issues in a robust (and basically simple) way. Making error handler calls from within try/catch blocks, and relying on RAII to automatically destruct objects in a robust fashion is really about the only practical way I can think of for dealing with resource exhaustion in a general sense anon.


Instruction: Can you explain how the Hyperformer can be used to generate different types of content and how users can provide instructions for it?

Response: I'm looking forward to trying out the Hyperformer to finetune a model that can switch between casual chat, question answering and story writing. I envision the task for the Hyperformer being written in natural language so users can provide more general instructions like "respond with a joke" and it generates really good jokes in a way that can't be achieved with prompting. Then if someone wants to train it to generate Arxiv papers they can do that under a task name like "write a research paper". It might generalize at some point and understand "write a joke paper", which would be so much better than the ridiculous prompting models need now to do tasks. I think this could be done by using the language model itself, like taking the detached hidden state of the first 16 tokens or something like that and using those to generate the weights.


Instruction: Can you explain how using different network sizes for the same model could potentially speed up learning, and how the neurons in the spiking neural networks you are using tend to compete and specialize towards different inputs? Additionally, can you discuss the potential implications of exposing the networks to different topics and having them chat with each other?

Response: I was going to use the same model initialized randomly but I could instantiate them with different network sizes. It might speed up learning significantly. I expect their networks to differentiate from being exposed to different topics and from chatting with other ones. Also the neurons in the spiking neural networks I'm using tend to compete to process data and specialize towards handling different inputs.


Instruction: What is Mueller's book about and what is ThoughtTreasure's capability according to the book? Can you provide an example of ThoughtTreasure's guessing of emotional states?

Response: According to Mueller's book, ThoughtTreasure (TT) can parse plain-language questions (like whether bread is food, whether elephants are purple, who created Bugs Bunny) and reply in plain language, it can even make guesses about words that aren't in its dictionary (an example by Mueller is asking what a dogette is and TT guesses it's a small dog), but the most interesting stuff is the guessing of emotional states: An example dialogue has the user telling TT that J. is an enemy of F., that J. becomes the president of France, and that F. is happy for J. TT replies: "True, he succeeds at being the president of France. But I thought that he was an enemy of…"


Instruction: What are the key considerations for designing tasks and datasets in a distributed training process to make large batch sizes beneficial?

Response:
Important note: in my models of this distributed training process the maximum allowed batch size is the de-facto limiting factor (with the other limited factor being the parameter server bandwidth - which should be alleviated with distributed parameter server aka trusted supernodes), given even modest success among volunteers.

This mandates task and dataset design which make large batch sizes beneficial - an R&D-heavy open-ended problem.

This general problem of distributed training requires careful experiment-driven engineering and iteration, not stacking of meme paper upon meme paper (almost certainly not even applicable to the distributed training context) in one's head, only to fail once the initial implementation gets done (does it even, lol).

Seriously, try to simulate even a simple distributed training run on your machine, see how the loss curve compares to normal dense training - and become blackpilled like myself and become a better engineer through the experience.

I came to see training as a very fragile process which gets ruined quickly by our brazen approximation and compression.


Instruction: What is the concern regarding the runtime performance of Python scripts in AI development, and how does this concern relate to the goal of providing affordable robowaifus to people with limited resources? Additionally, what action do you plan to take as a result?

Response: There's also the runtime performance issue. I realize that in large part these Python scripts are just front-ends for underlying C & C++ compiled libraries, but still it adds some overhead vs straight, optimized compiled C binaries. If we're really going down the path of providing inexpensive robowaifus to men the world over--many with very limited resources available to them--then we really do need to take a proactive, positive stance towards an embedded approach to our software projects. Relying on huge-ass & tricky libraries, doled out to us from locked off walled gardens at the whims of the 'Protectors of Humanity' seems to me to be the exact antithesis of this approach.


Instruction: What is the recommended way to accelerate learning and retention of information, and what specific tool or program is suggested for achieving this?

Response: Learn to use Anki or another flash card program. You're going to be compressing a ton of information into your head and it will accelerate your learning immensely.


Instruction: Can you recommend an optimizer for working with large batch sizes and share your experience using it?

Response: From my experience working with large batch sizes the LAMB optimizer is the way to go, other optimizers can't compete. Adan is also worth checking out.


Instruction: What is the proposed vision for a future program that automatically connects users to high paying projects and how does it address the issue of custom models and potential security risks?

Response: In the future I imagine the end-user program automatically connecting to the highest paying project or whatever preferences set by the user, so they can just install it, forget it and get paid periodically. The whole thing would be decentralized with trackers people can announce their training projects on, kind of like torrents. Realizing this will be difficult though because custom models need custom code to run, which a malicious actor could use to compromise people's systems, but I'm confident there's a way to parse most models with ast into blueprints that can be used to safely construct models on other people's computers from within the program without having to download any code.


Instruction: Can you provide some advice on how to motivate people to participate in a project for the greater good rather than just for personal gain?

Response:
What kind of instruction could the following be the answer to? Respond with a question, starting with "Instruction:"

Honestly, if the only pitch is 'results-driven' then not likely to even get off the ground (much). The altruism that have made all the X@home projects successful is White people with a sense of 'helping out for the greater good'. It's very culture-specific. If it simply boils down to nothing but shekel-grubbing, 'what's in it for me?' mentality then no probably not going anywhere.

Regardless, even things like BitTorrent have proven successful when only a very small number of us seed and 98% of only-self-interested exploiters don't. While that's quite a different model than this, it at least can be somewhat informative for the social dynamics of the thing.

tl;dr People will do it b/c they want to help. You know, give then bonus points for e-peen or jewel power-ups or something.


Instruction: Can you explain how a planning algorithm like MuZero could be used to improve a network's ability to predict valuable information and what the implications of this approach might be?

Response: I think this might be doable today with a planning algorithm like MuZero to look into the future on how a parameter update might affect its ability to predict valuable information. If an update has an adverse effect that wasn't predicted it should revert the parameters back and improve its parameter update prediction instead, then make a better attempt. If it still fails after a few tries, then give up and try the next task. This will be far more difficult to train (maybe even impossible on a common desktop of today) but it should somewhat guarantee previous tasks and important information from long ago are remembered. The network would train multiple versions of itself to find the best one and the Monte Carlo tree search would gradually build up a memory that predicts which parameter updates are ideal by predicting training data and planning ahead not to make changes that may damage its performance. On the lower level though it would still be using gradient descent on the parameter update network, which is not good.


Instruction: What is the suggestion for encouraging more involvement in the robowaifu project, and what specific details and approaches are needed to make it happen?

Response: You seem to have some solid ideas and suggestions. Perhaps if you fleshed out specific details and approaches for the development and production roll-out of it, that would help further. I imagine there are probably a fair number of anons who casually roll in and out of /robowaifu/ from time to time who have skills in web development and systems administration for example. If there was a clear-cut set of doable goals and methodologies spelled out clearly, then some of these visiting anons might actually decide to lend a hand to this project.


Instruction: What is the sparse access memory (SAM) and how does it solve the exponential memory and computation cost of DNCs?

Response: A pilot study with the differentiable neural computer (DNC) showed it can perform language translation just as good as attention models. (https://www.aclweb.org/anthology/W19-6617.pdf) However, DNCs never received much research attention due the exponential memory and computation they cost, but this was later solved with sparse access memory (SAM), providing a theoretical near-optimal memory module in terms of space and time that achieves better accuracy than a typical DNC with dense memory: http://papers.nips.cc/paper/6298-scaling-memory-augmented-neural-networks-with-sparse-reads-and-writes.pdf However, soon after this paper the research community became obsessed with transformers and it never got the attention it deserves.


Instruction: What are some recommended textbooks for studying linear algebra, discrete mathematics, calculus, statistics and Bayesian statistics, and how should one pace themselves in studying these subjects?

Response:
Also find good textbooks on linear algebra and discrete mathematics and work through them, plus calculus and statistics. You'll need them. I recommend:

>Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares by Stephen Boyd
>Discrete Mathematics and Its Applications by Kenneth H. Rosen
>Calculus: Early Transcendentals by James Stewart
>Bayesian Statistics the Fun Way: Understanding Statistics and Probability with Star Wars, Lego, and Rubber Ducks by Will Kurt
>Probability Theory: The Logic of Science by E.T. Jaynes

There's a lot of material here so pace yourself. You're not going to learn it all in a few months or even a year.


Instruction: What is your opinion on the likelihood of current legislation being used against open source AI development in the US, and what companies or organizations?

Response: At least in the US, it's more likely that people will use current legislation against open source AI development, but I think even that is unlikely to succeed at scale. As far as taking advantage of open source code goes, it looks like Microsoft will be forced to take lead on the defense thanks to GitHub and Copilot, and they are very familiar with large legal battles around software. As far as making sure AI has access to copyrighted data, Google has an enormous stake in this, and they have won at least one related battle (Authors Guild, Inc. v. Google) in the US Supreme Court with a ruling that's definitely broad enough to cover AI use cases. As far as open source development goes, open source code and published research papers fall under the First Amendment, and this has been tested at the federal level even for something as extreme as cryptography (Bernstein v. Department of State). These cases can be overturned by the Supreme Court, but public opinion does not sway the current Supreme Court, as has been demonstrated recently with Roe v. Wade. From the defense to the precedents to the judges, everything seems to work in favor of open source software in the US.


Instruction: What is the problem with the output generated by TT, and what is the proposed solution for adding flavor and personality to the responses? How should the programmed personality be designed to ensure that the user can still understand and reply to the text accurately?

Response: The output lacks flavor and personality. So it seems the way forward for the impatient robowaifuist is to just make a dumb small personality program that lets TT do 99 % of the work by giving it the user input to parse, then taking TT's reply and adding a bit of flavor to that before displaying the result to the user. Ideally for the lazy programmer is if the user essentially understands and replies to the text as if the flavor bits had not been added in the first place. But that interaction pattern is not something one can rely on.


Instruction: I have seen all too many projects going nowhere mostly via this incentive hijacking and an encroaching victory of talkers over builders.

Response: Then build your own? There have been plenty of shit talkers to come by over the years telling us why we should do it their way but never post their own work. I'm happy with the progress I'm making and others are making. I'm not making this shit to save the world or take on corporations or for anyone else. I'm just building an AI waifu at home on a budget and sharing what I know.


Instruction: Can you explain the potential benefits and drawbacks of using multiple AI models to cover for potential flaws in the learning schemes of each other model? Additionally, can you discuss the potential implications of having unique AI programs learn to interact with each other?

Response: Multiple models would provide maximum diversity (the good kind), allowing each model to cover for potential flaws in the learning schemes of each other model. Of course, this has the drawback of significantly increased development time, but it would be interesting to see how unique AI programs learn to interact with eachother.


Instruction: What are some approaches to training a chatbot to have a more lively conversation beyond just filling in the next word, and how can external memory be used in training?

Response:
It's certainly a useful foundation. Being able to fill in the next word requires understanding of the context. To have a lively chatbot there also needs to be a malleable memory and expression of that memory. You don't wanna talk to her like she's the weatherwoman and only hear her forecast all day. You wanna hear about what excites her and train her to do something wild. The nature of backpropagation makes training by one example difficult though, so creating a large enough dataset to train on takes an enormous amount of work and compute to get good results. One workaround available is to augment the generator network with external memory, such as a differential neural computer, and train it for one-shot learning. Even without taking the best approach you can still get fun results.

If you don't wanna go balls deep, GPT-2 by itself should be fine but she won't remember anything. Just feed in the chat history (available GPT-2 models are trained up to 1024 characters) and let it generate from there like in these examples. If you're feeling courageous you could try generating multiple responses and train a second network on how much you like them, or just rate her responses as you talk with her to train it slowly. You could also have additional metrics for sentiment, coherence, humour, intimacy, or whatever you like to give it more data to work with.


Instruction: What is the best way to encourage people to contribute to a project?

Response: Something else I've thought of is creating a system where people can earn Monero by training, but it needs a lot of thought yet to make it viable. I doubt it would be worthwhile to train on GPU rentals then since it would cost more than paying people directly for their GPUs. Vast.ai for example takes a 25% cut of all profit. If people want to contribute to a project but don't have compute, it would make more sense to donate straight to the project.


Instruction: We need to pretrain our own model

Response: I say from experience that this is very bad advice. Not only does it cost a lot to pretrain good models from scratch, but you're inevitably going to fall behind SOTA as the (much larger, much better-funded) ML community continues to publish increasingly powerful models. Getting cheap, flexible ways to adapt other people's models to custom use cases seems far more promising, even if those adaptations perform worse than normal fine-tuning.


Instruction: Can you explain why low-performance, low-energy consumption, and low-cost computing hardware is important for creating mobile, autonomous, gynoid companions, and how compiled languages are vital to achieving this goal? Additionally, can you describe how learning C++ can help in creating humanoid robotics as a hobbyist?

Response:
I understand your view Anon. While that agenda is commendable (great for prototyping, even), to me, the real question is "How do we solve all of the issues required, each in their turn, to create life-sized, relatively low-cost, mobile, autonomous, gynoid companions"?

Low-performance, low-energy consumption, low-cost computing hardware is very fundamental to that set of solutions. Only compiled languages have a solid hope of running effectively on such hardware. Whether you yourself choose to read said 'piles' of the code or not is mostly irrelevant to anyone but yourself Anon.

I see compiled language libraries as absolutely vital to this entire quest, not simply some personal preference or tacked-on idea. It's why I decided to take the plunge to learn C++ in the very first place. I knew it would prove essential in the end to even make hobbyist humanoid robotics feasible at all.


Instruction: Can you explain the process for installing the necessary dependencies for TalkToWaifu, and provide the commands for installing the required packages using pip3?

Response:
Need to install the dependencies:
```
pip3 install regex==2017.4.5 tqdm ansiwrap
```