Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I made a transformer to predict a simple sequence manually

vgel.me

Post reply on HN
Print view View original

91–100 of 104 posts

Re: I made a transformer to predict a simple sequence manually

#91

Post by GaggiX »

seethedeaduu wrote: post #85
GaggiX wrote: post #84

Earlier quoted context omitted.

[flagged]

Gender-affirming care, also known as transition (transition what? Their sex!), is done in order to make trans people more comfortable in society, in their inner self (hormones affect mood, behavior, feelings, etc) and in their body by changing their sex. This is why they are prescribed sex hormones as these are what guides gene expression and decide sex-dependent characteristics. While some trans people would be cont…

Gender-affirming care and gender transition (as the results on Google will call it if you accidentally search "sex transition"), none of this changes your sex, most mammals (including humans) have the XY sex determination system, taking estrogens or other hormones doesn't make you lose your Y and gain an extra X, nor does it make you start producing ova, the only difference is how you look and sound, which (hopefully) makes you more comfortable with your gender, and that's why it's called "gender-affirming care" and "gender transition".

Re: I made a transformer to predict a simple sequence manually

#92

Post by GaggiX »

seethedeaduu wrote: post #87
voop wrote: post #86

Earlier quoted context omitted.

[dead]

We are not discussing whether sex is a spectrum or not though. Was the nonbinary person that you talked with undergoing medical transition? If not this might explain their view.

(you are talking with another person, not me)

Re: I made a transformer to predict a simple sequence manually

#93

Post by gbear0 »

amilios wrote: post #38
Supply5411 wrote: post #37

I've been kicking around a similar idea for awhile. Why can't we have an intuitive interface to the weights of a model, that a domain expert can tweak by hand to accelerate training? For example, in a vision model, they can increase the "orangeness" collection of weights when detecting traffic cones. That way, instead of requiring thousands/millions more examples to calibrate "orangeness" right, it's accelerated by a…

The technical reason it can't be done (or would be very difficult to do) is that weights are typically very uninterpretable. There aren't specific clusters of neurons that map to one concept or another, everything kind of does everything.

On the idea of interpreting the weights, I've been very interested if it's possible to compute basis vectors of the weights matrix to define the core concepts within the model and then do a change of basis to allow reorganizing the model to more human understood concepts?

I think the inherent compression of a specific training set into a matrix makes this more difficult cause the basis vectors likely won't contain clean representations of human ideas, but I also wonder if starting a new training set with an initialized (or fixed) matrix of human defined concepts would help align the model's weights to something that can be interpretable

Re: I made a transformer to predict a simple sequence manually

#94

Post by GaggiX »

seethedeaduu wrote: post #83
GaggiX wrote: post #66

Earlier quoted context omitted.

[flagged]

I don't think they undergo hormonal treatment just to transition their gender.

Well, one comment at the root was flagged and the thread was deleted, so hopefully I can reply to you here, plus you were starting to replying to another guy you thought he was me.

My answer was: "Gender-affirming care and gender transition (as the results on Google will call it if you accidentally search "sex transition"), none of this changes your sex, most mammals (including humans) have the XY sex determination system, taking estrogens or other hormones doesn't make you lose your Y and gain an extra X, nor does it make you start producing ova, the only difference is how you look and sound, which (hopefully) makes you more comfortable with your gender, and that's why it's called "gender-affirming care" and "gender transition"."

Re: I made a transformer to predict a simple sequence manually

#95

Post by seethedeaduu »

GaggiX wrote: post #94
seethedeaduu wrote: post #83

Earlier quoted context omitted.

I don't think they undergo hormonal treatment just to transition their gender.

Well, one comment at the root was flagged and the thread was deleted, so hopefully I can reply to you here, plus you were starting to replying to another guy you thought he was me. My answer was: "Gender-affirming care and gender transition (as the results on Google will call it if you accidentally search "sex transition"), none of this changes your sex, most mammals (including humans) have the XY sex determination s…

I don't understand the focus on genes that some people seem to be placing. After the gonads are formed the Y chromosome does not really matter (unless if you have a particularly nasty mutation). All Y does is carry the SRY gene. If you did magically managed to lose your Y chromosome would that make you a woman? No! You would stay a man and literally nothing would change in your life. Same goes if you have two X chromosomes and one magically gets replaced with a Y.

I am also worried about the focus on genes given the existence of intersex people. It's especially weird since sex categories existed long before the creation of modern biology and at least for some people the term sex got redefined to exclude certain men and women who often don't even realize it.

The term gender affirming care seems accurate to me, as part of affirming their gender, trans people (who used to be called transsexual) attempt to transition so that their sex matches their gender identity. On the other hand, the term gender transition seems to include both sex changes and social transition.

I should also mention that estrogen when does not change how you sound, it does however cause other changes not related to one's appearance.

Re: I made a transformer to predict a simple sequence manually

#96

Post by mistrial9 »

PoignardAzur wrote: post #82
Supply5411 wrote: post #37

I've been kicking around a similar idea for awhile. Why can't we have an intuitive interface to the weights of a model, that a domain expert can tweak by hand to accelerate training? For example, in a vision model, they can increase the "orangeness" collection of weights when detecting traffic cones. That way, instead of requiring thousands/millions more examples to calibrate "orangeness" right, it's accelerated by a…

The reason you're looking for is called "The Bitter Lesson". The short version is, trying to give human assistance to AIs is almost always less cost-effective than making them run on more computing power. By the time your human expect has calibrated your weight layers to detect orange traffic cones, your GPU cluster has trained its AI to detect traffic cones, traffic ligts, trees, other cars, traffic cones with a sli…

"The bitter lesson of machine learning is that building knowledge into agents does not work in the long run, and breakthrough progress comes from scaling computation by search and learning.02 This applies to domains where domain knowledge is weak or hard to express mathematically. The rapid progress of ML applied to LQCD, mol.2 dyn., protein folding, and computer graphics is the result of combining domain knowledge with ML."

This passage says that the advantages of scaling a certain kind of learning are especially good when .. two conditions.. but a side-effect of that statement is, when knowledge is well known, and maybe straightforward to express, these kinds of learning systems, not as great.. this is true.

without taking on big other topics, I think this "bitter lesson" is unspecific enough to include some self-serving utility.. just tell the other camps to give up, you lost. that sort of thing.

Re: I made a transformer to predict a simple sequence manually

#97

Post by GaggiX »

seethedeaduu wrote: post #95
GaggiX wrote: post #94

Earlier quoted context omitted.

Well, one comment at the root was flagged and the thread was deleted, so hopefully I can reply to you here, plus you were starting to replying to another guy you thought he was me. My answer was: "Gender-affirming care and gender transition (as the results on Google will call it if you accidentally search "sex transition"), none of this changes your sex, most mammals (including humans) have the XY sex determination s…

I don't understand the focus on genes that some people seem to be placing. After the gonads are formed the Y chromosome does not really matter (unless if you have a particularly nasty mutation). All Y does is carry the SRY gene. If you did magically managed to lose your Y chromosome would that make you a woman? No! You would stay a man and literally nothing would change in your life. Same goes if you have two X chrom…

XY sex-determination system is based on the fact that the chromosomes one has at birth cannot change naturally throughout their lifetime, so magical or hypothetical scenarios where chromosomes can be altered doesn't align with our current scientific understanding, but if it makes you happier, sure, in a magic world humans would be able to change their sex.

Also, I don't see how the existence of intersex people should be somewhat problematic in this regard, intersex is a congenital condition, like people who are born with six fingers, that doesn't make the fact that humans have 5 fingers controversial.

To be honest I think I'm kind of stating the obvious and most people (trans people included obv) would agree with me, which is why they use "gender affirming care", "gender transition" and "transgender" (instead of transsexual which is in fact seen as an outdated term). If you don't care about true biological data (plural of datum), but only the opinions of trans people, I can ask my trans friends (one nonbinary and two trans women) what they think about it, even though I actually know them well, and they would agree with me.

Re: I made a transformer to predict a simple sequence manually

#98

Post by fransje26 »

thomasahle wrote: post #78

I thought I understood transformers well, even though I had never implemented them. Then one day I implemented them, and they didn't work/train nearly as well as the standard pytorch transformer. I eventually realized that I had ignored the dropout, because I thought my data could never overfit. (I trained the transformer to add numbers, and I never showed it the same pair twice.) Turns out dropout has a much bigger…

Would you have any literature to recommend, to help get some insight in how to approach the task?

Re: I made a transformer to predict a simple sequence manually

#99

Post by seethedeaduu »

GaggiX wrote: post #97
seethedeaduu wrote: post #95

Earlier quoted context omitted.

I don't understand the focus on genes that some people seem to be placing. After the gonads are formed the Y chromosome does not really matter (unless if you have a particularly nasty mutation). All Y does is carry the SRY gene. If you did magically managed to lose your Y chromosome would that make you a woman? No! You would stay a man and literally nothing would change in your life. Same goes if you have two X chrom…

XY sex-determination system is based on the fact that the chromosomes one has at birth cannot change naturally throughout their lifetime, so magical or hypothetical scenarios where chromosomes can be altered doesn't align with our current scientific understanding, but if it makes you happier, sure, in a magic world humans would be able to change their sex. Also, I don't see how the existence of intersex people should…

Please help me understand this. You are in fact claiming that if somehow someone lost their Y chromosome that would make him a woman despite the lack of any physical or mental changes? I think you have a radically different view of sex compared to most people if so as it is fundamentally incompatible with our social structure.

The claim that most humans have 5 fingers or that most men have XY chromosomes is not controversial. However the claim that you must have 5 fingers to be a human is! This is what the XY sex-determination system is based on, they noticed that most men have XY chromosomes and that most women have XX chromosomes and declared intersex men and women who do not match this model to actually be of the other sex by making an inaccurate map from genes to sex (which is perceived socially and existed before said model).

I believe that the above 2 arguments together are enough to show that it is not an accurate nor a useful sex determination system from a logical perspective.

I think people nowadays avoid the term transsexual because it might be confused with a sexual orientation. I also feel like some people consider the term sex to be vulgar, which is why even these who consider gender=sex often prefer the term gender even when it does not refer to the grammatical gender. Finally the term transgender is often used by some in order in order to be inclusive to people who are not medically transitioning.

You are free to show this thread to your friends if you want. I am sure there are trans people in both camps of the argument.

Re: I made a transformer to predict a simple sequence manually

#100

Post by GaggiX »

seethedeaduu wrote: post #99
GaggiX wrote: post #97

Earlier quoted context omitted.

XY sex-determination system is based on the fact that the chromosomes one has at birth cannot change naturally throughout their lifetime, so magical or hypothetical scenarios where chromosomes can be altered doesn't align with our current scientific understanding, but if it makes you happier, sure, in a magic world humans would be able to change their sex. Also, I don't see how the existence of intersex people should…

Please help me understand this. You are in fact claiming that if somehow someone lost their Y chromosome that would make him a woman despite the lack of any physical or mental changes? I think you have a radically different view of sex compared to most people if so as it is fundamentally incompatible with our social structure. The claim that most humans have 5 fingers or that most men have XY chromosomes is not contr…

I think I was very clear when I said: "XY sex-determination system is based on the fact that the chromosomes one has at birth cannot change naturally throughout their lifetime, so magical or hypothetical scenarios where chromosomes can be altered doesn't align with our current scientific understanding", if people manages to change their chromosomes I guess the sex-determination system would probably change but we don't live in that magical world so things are much easier (yeah!)

>However the claim that you must have 5 fingers to be a human is! This is what the XY sex-determination system is based on

No idea what you're talking about ahah, people with a congenital condition are still human and no one said otherwise.

Post reply on HN