This is thread is for discussing general tech related algo-trading topics.
Hijacking this thread with MQL programming solicitations will get you banned!
Hijacking this thread with MQL programming solicitations will get you banned!
You talk the talk I walk the talk 8 replies
Much of Forex talk is illusional esoteric talk 141 replies
General Motor's Influence 2 replies
VTTrader: General Discussion 24 replies
Beginner question about MT4 and trading in General. 4 replies
Quoting VEEFX
{quote} HI there again, I am desperate to get out of the MT4/5 retail brokerage space and go the FIX route. I am zero'ng on Python to replace my EA automation so research to build a solution in parallel has begun. Is there a better alternative to Python? I came across this but haven't played around much. I would also like some GUI frontend to make things easier. My order execution piece is pretty straightforward but the administrative piece (management functions like DD, margin, trade, RM, exposure...
The entire MQ4 code can be converted to C++.On my system under C++ the CPU runs 0% with 350 threads at 1ms sleep time.MT4 is killing CPU's because using WinApi for draw functions not because of the math calculations and isn't even allowing multithreading.More colors text or draw on your charts more CPU needed.
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD
Quoting masterx584
{quote} The entire MQ4 code can be converted to C++.On my system under C++ the CPU runs 0% with 350 threads and 1ms sleep time.MT4 is killing CPU's because using WinApi for draw functions not because of the math calculations and isn't even allowing multithreading.More colors text or draw on your charts more CPU needed.
I agree. It's the single threading limitation... and I use no charts and no technical indicators so I know it is not my graphics adapter. I learnt C in the 80s in high school and then some more for my post-graduate project to develop a compiler for a language. Is there a mq4 to C++ convertor that I can play with?
EDIT: Its the 9 levels deep nested recursive logic which hogs my CPU on every tick
Attached Image
https://www.forexfactory.com/attachm...1&d=1534973088
Quoting VEEFX
{quote} HI there again, I am desperate to get out of the MT4/5 retail brokerage space and go the FIX route...
OK, this is going now completely off-topic here but I'll try to give some general answers.
Language is irrelevant, simplicity and ROI are important (KISS). If you'r already familiar/professional with one language don't switch (as long as it's not VB or Excel). Plan and measure in *your* time, that's the resource you can control.
FIX is the standard for institutions but it's an unimportant detail for retail. Good to know but not important. You don't need FIX to become consistently profitable and you will not increase your chances to do so by using it. The opposite is true. What really matters is what stage you are on with your trading.
Becoming broker/platform independant is a natural wish, especially as MetaQuotes will cease to exist as we knew it. What you need for this depends on your strategy (if you have one). The options we have in the future in retail space are platforms like cTrader, Quantower etc. You can setup a broker independant MT4 for charting and market analysis if you want to, even if there's no MetaQuotes anymore. But if you don't need charting but only market scanning it will be completely different. May be you understand now that there can be no general answer as it really depends on your needs. Let's say we don't have a strategy and need a good charting module. OK, we might get used to MT5 for this (this is all MT5 is good for). Retail more than others needs to be able to adapt.
The real issue with retail automation is not finding a profitable strategy. The real issue is managing the infrastructure and potential infrastructure errors. A realistic backtest showing 7% return per month over 5 years will not work out as expected in a live account, as your account will crash once or twice a year because you have a network outage or made a stupid coding mistake. There is not a single code example on mql5.com or forexfactory.com with proper error handling. Think about this if you have experience in IT. On the other hand there is always room for unexpected error. Problem here: you can test your strategy but you cannot test in a backtester (whatever platform) for those unexpected errors.
So in summary the real issue is not what platform/language do I use. The real issue is how do I prepare my trading for the...let's call it Black Swan events. They will always come and they are the real threat. It might be your fault, it might be a 15th of January 2015. It might turn out that I better control such risk by manual trading (this is what institutions do). Manual trading also makes you independant of platform changes. But it requires completely different skills which a retail trader can aquire only in a very difficult way.
So focus on what you already can do and simplify that. Don't complicate it by introducing more abstractions (e.g. FIX). At the current moment in time there is no need for FIX in the retail space. I say "need"...
1
Quoting VEEFX
{quote} I agree. It's the single threading limitation... and I use no charts and no technical indicators so I know it is not my graphics adapter. I learnt C in the 80s in high school and then some more for my post-graduate project to develop a compiler for a language. Is there a mq4 to C++ convertor that I can play with?
Only the hard way as i know.U need to add Winsock for connections , a draw engine for C++ environment ,to rewrite all the algorithms or to create new multithreaded ones.. to allocate memory etc. c++ have alot of elasticity and can compile assembly language inside ,i have a few functions in machine code for more speed.It's easy for me to speak now but i worked almost 2 years on the entire thing ...mostly on my multithreaded algorithms.Anyway if u start to build your C++ system i can help u with things like multithreaded clients,graphics ,how to build charts or how to transform functions like NormalizeDouble which is pain..u can almost create a copy of MT4 ...how to completely eliminate the netwotk lag.
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD
Quoting VEEFX
Its the 9 levels deep nested recursive logic which hogs my CPU on every tick
Never do this. A 9 level deep loop will kill every performance, even if it's written in Assembler.
Quoting VEEFX
{quote} I agree. It's the single threading limitation... and I use no charts and no technical indicators so I know it is not my graphics adapter. I learnt C in the 80s in high school and then some more for my post-graduate project to develop a compiler for a language. Is there a mq4 to C++ convertor that I can play with? EDIT: Its the 9 levels deep nested recursive logic which hogs my CPU on every tick {image}
U have a 9 level big problem ...break it into multiple small problems ...go multithreading.
Attached Image (click to enlarge)
https://www.forexfactory.com/attachm...1&d=1534975667
Right now i am with only 123 threads ,8 clients..16 threads *client,used to connect 20 clients before and 340+ threads.I calculate tick by tick data no timed bars.My threads are setted to 1ms sleep.They keep pushing.The algorithm alone divided in multiple cells running through threads is 3000+ code lines totally.It's more about how u divide & conquer.
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD
OK, we already hijacked the thread. Sorry to the OP.
Key to becoming better/more independant/more successful is not improving our platform or an indicator or a strategy.
Key is awareness. Awareness *why* the natural consequence of our trading approach as a retail trader is for the vast majority failure. If we gain that awareness we are able to make the required changes. Without that awareness we are doomed. A platform or language will not help us.
Attached Image (click to enlarge)
https://www.forexfactory.com/attachm...1&d=1534976458
As Morad Askar says: The statistics about 99% of failing retail traders are correct. But it is a useless statistic. Online trading has no entry barriers, a 14 year old can open an account, upload the ID of his sister which lays around in her room all day long and fund the account with the credit card of his mother which lays around in the kitchen.
Take a chirurg with proper study and years of experience. Take 100 people you find randomly in the streets and give each one of them a scalpell. What percentage of the 100 people do you expect to become successfull chirurgs?
None. 100 percent failure. Is this statistic correct? Yes. Does it correctly depict your chances to become a successfull chirurg? Of course not.
It's the same with trading. Awareness is key.
Inserted Video
Quoting masterx584
{quote} U have a 9 level big problem ...break it into multiple small problems ...go multithreading. {image} Right now i am with only 123 threads ,8 clients..16 threads *client,used to connect 20 clients before and 340+ threads.I calculate tick by tick data no timed bars.My threads are setted to 1ms sleep.They keep pushing.The algorithm alone divided in multiple cells running through threads is 3000+ code lines totally.It's more about how u divide & conquer.
Multithreading aids in IO bound, flat, and asynchronous operation. For CPU intensive tasks, multiprocessing is required. If the algorithm (actually) requires nested loops then neither would be useful since the each inner loop is interdependent on the iteration outcome of its outer loop. I've rarely ran up against a problem that couldn't be flattened, but I won't presume to know the logic in question, and assume that OP knows what he/she is doing.
Quoting emmzett
OK, we already hijacked the thread. Sorry to the OP. Key to becoming better/more independant/more successful is not improving our platform or an indicator or a strategy. Key is awareness. Awareness *why* the natural consequence of our trading approach as a retail trader is for the vast majority failure. If we gain that awareness we are able to make the required changes. Without that awareness we are doomed. A platform or language will not help us. {image} As Morad Askar says: The statistics about 99% of failing retail traders are correct. But it...
hahah, u think i was so stupid to loose 2 years to build and put my algos into a completely isolated and secured network if they weren't valid?..away from any broker spy plugins ... with OP hijacked most of the services disabled completely invisible against Kali Linux or Parrot Security tested in the same network...i think who will work harder will be more succesfull in life.Who want to take it professionally need to cut brokers from this ecuation with all the services they offering and all the preparation or webinars...What u put is what u get...as allways.. and i am able to take trades without never watching the Bid...only the spread.Peace .
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD
Quoting emmzett
{quote} Never do this. A 9 level deep loop will kill every performance, even if it's written in Assembler.
Thanks emmzett and masterx for the responses. I really appreciate it. Level 9 deep is only on one pair..... the depth also exponentially increases depending on when my signal confluence happens which triggers a recursive crawl if you know what I mean but I have capped it at 32 levels. It's is designed as a sequential crawl and cannot be done via parallel processing so to mimic multi-thread, I run it across several terminals plus one terminal to handle all portfolio/trade mgmt functions but still my CPU clocks continuously non stop at 50%-60%. Ultimately, I need to use Spot to trade regulated futures, indices, sub-sector ETFs etc.... any market that is a derivative of underlying instruments as this model is quite expandable to other markets (because it is purely data driven and contained within the weekly candle so not a whole lot of volume of data involved) so yeah... i do have a strategy. I just do not want reliance on a broker-provided platform. Order Execution Speed/latency/slippage etc is not my concern at all as they have no or minimum impact on my overall outcome. What's important is the timely processing (computations) to complete within one minute for the signal to remain valid.
Hi Guys,
This must be very interesting conversation, but it is not related to FIX API.
It would be more beneficiary for you to start your own thread. Also for us who want to discuss FIX API.
Thanks
Quoting VEEFX
{quote} Thanks emmzett and masterx for the responses. I really appreciate it. Level 9 deep is only on one pair..... the depth also exponentially increases depending on when my signal confluence happens which triggers a recursive crawl if you know what I mean but I have capped it at 32 levels. It's is designed as a sequential crawl and cannot be done via parallel processing so to mimic multi-thread, I run it across several terminals plus one terminal to handle all portfolio/trade mgmt functions but still my CPU clocks continuously non stop at 50%-60%....
My initial computation takes 30 sec too but i am able to pack that data in a way that allows me to save it on HDD & to reutilize it with the new arrived ticks as it is without any other computations.I am only making ticks based calculations no timeframed bars.I create my own "smart packets" that can be utilised as it is.I pack and create my own history ... don't know the mkt price for none of my trading pairs... am only based on the market value *strength, no price action,high lows or whatever.I don't care what price it has but only if the market is into a value*strength accumulation range or is going to explode after being ranging for a while.I can wait 2-3 weeks to take a pair outside of the range going to replace a top trending one who's weakening and ready to get into accumulation.Is allways an exchange.
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD
1
Quoting zosoforever
Hi Guys, This must be very interesting conversation, but it is not related to FIX API. It would be more beneficiary for you to start your own thread. Also for us who want to discuss FIX API. Thanks
That's true.I'm very sorry.I'm done.See u guys.
DislikedSubscribed "I am desperate to get out of the MT4/5 retail brokerage space and go the FIX route. I am zero'ng on Python to replace my EA automation so research to build a solution in parallel has begun. Is there a better alternative to Python? I came across this but haven't played around much." I got out ..it's a totally diff world.Escaped from MT4Alcatraz.Ignored
DislikedMe too...but I am ex-geek and a old dawg who left systems engineering about two decades ago to go on the business side of the equation :- {quote} Regarding your last post on stoding data set on HDD, I did that route too but disk IO was killing me more and I could not risk missing new ticks or getting data stale...plus my EA can run on one pair one timeframe if I really want to spit out the overhead but I still like a single EA approach to free up my time was the worrying about multiple instances. Could you expand on what you mean by 'market value"...Ignored
Disliked{quote} In my vision the entire mkt stocks,currencies etc. ,all the mkt participants have something in common ==>USD .This lead me to build a system where i can give a certain level of value *strength to any mkt participant. I can compare for example Apple or Amazon against NZD or CAD even if apparently they have nothing in common.I almost can smell the buying selling accumulation pressure inside my "value accumulation period"... am focusing right now on the currencies exclusively. I am bridging all the mkt participants through USD and i bet most...Ignored
Disliked{quote} Interesting concept. USD is indeed King. I could ask more questions on how you measure value and relative strength but OP will probably want to focus on tech talk. Did you mean range instead of period in "value accumulation period" ?Ignored
DislikedMe too...but I am ex-geek and a old dawg who left systems engineering about two decades ago to go on the business side of the equation :- {quote} Regarding your last post on stoding data set on HDD, I went that route too but disk IO was killing me more and I could not risk missing new ticks or getting data stale...plus my EA can run on one pair one timeframe if I really want to spit out the overhead but I still like a single EA approach to free up my time without worrying about multiple instances. Could you expand on what you mean by 'market value"...Ignored
Disliked{quote} Value*strength range at micro micro level.Just close to JP Morgan bots.Every fire need to start from somewhere.I'm looking to find that spot + the direction of the wind.All the info is tick by tick based,encoded.. but we do our best.Ignored
Disliked{quote} Elaborated data for the past 45 ticks range/duration/scale as shown in your screenshot? I think I understand. What's your threshold for directional trigger ?Ignored
Disliked"I am desperate to get out of the MT4/5 retail brokerage space and go the FIX route. I am zero'ng on Python to replace my EA automation so research to build a solution in parallel has begun. Is there a better alternative to Python? I came across this but haven't played around much." I will make you a simple question. Are you in this business for earn money or for burn your life time?Ignored
Disliked{quote} This question doesn't translate well to English. What is this suppose to mean?Ignored
Disliked{quote} If you want earn money trading don't seem the best optionIgnored
Disliked{quote} I think it's a problem with the translation. I think he means to say that if you want to make money in trading, then porting your strategy over to a new language isn't worth the time investment.Ignored
Disliked{quote} Just if spend a very long time in that kind of project compensate you. If you want earn money trading don't seem the best optionIgnored
Disliked{quote} Hi braintheboss, I am very interested for FIX. Did you mean that it takes too much time to setup everything for FIX trading? ThxIgnored
Disliked{quote} Complex projects are expensive in time. But i wonder if his strategy works ( the current tools he is using ) then really he needs replace a full platform?Ignored
Disliked{quote} I see, you are talking about VEEFX's strategy, not overall for FIX trading.Ignored